Method and apparatus for encoding/decoding image

ABSTRACT

An image encoding method includes configuring block division information and size information of a boundary block which is located at a boundary of an image and is smaller than a size of a basic encoding block, on the basis of size information of a picture and size information of the basic encoding block; dividing the boundary block into at least one encoding block, on the basis of the size information of the basic encoding block and the size information and the block division information of the boundary block; and encoding the at least one divided encoding block.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.17/324,280 filed on May 19, 2021, which is a continuation of U.S.application Ser. No. 16/631,325 filed on Jan. 15, 2020 (now U.S. Pat.No. 11,044,475), which is a National Stage of International ApplicationNo. PCT/KR2018/007924 filed Jul. 12, 2018, claiming priority based onKorean Patent Application No. 10-2017-0090633 filed Jul. 17, 2017.

TECHNICAL FIELD

Example embodiments of the present invention relate to a method ofencoding/decoding an image having a size that is not a multiple of thesize of a basic encoding block, and more particularly, to a method andapparatus for encoding/decoding an image having a size that is not amultiple of the size of a basic encoding block by expanding the image oradjusting the encoding block.

RELATED ART

With the spread of the Internet and mobile terminals and the developmentof information and communication technology, the use of multimedia datais increasing rapidly. Therefore, in order to perform various servicesor tasks through image prediction in various systems, a need forimprovement of performance and efficiency of an image processing systemhas increased considerably, but a research and development result thatcan respond to the need has heretofore been insufficient.

As described above, in an image encoding and decoding method andapparatus of the related art, there is a demand for improvement ofperformance in image processing, and particularly, in image encoding orimage decoding.

DISCLOSURE Technical Problem

Example embodiments of the present invention provide a method forencoding/decoding an image having a size that is not a multiple of thesize of a basic encoding block.

Example embodiments of the present invention also provide an apparatusfor encoding/decoding an image having a size that is not a multiple ofthe size of a basic encoding block.

Technical Solution

In some example embodiments, an image encoding method includes: settingsize information and block split information of a boundary block that islocated on a boundary of an image and smaller than a basic encodingblock on the basis of size information of a picture and size informationof the basic encoding block; splitting the boundary block into at leastone encoding block on the basis of the size information of the basicencoding block and the size information and the block split informationof the boundary block; and encoding the at least one encoding block.

Here, the image encoding method further includes: expanding a size ofthe picture on the basis of the size information of the boundary blockand the size information of the basic encoding block; and adjusting thesize information of the boundary block on the basis of the expanded sizeof the picture.

Here, the splitting of the boundary block into the at least one encodingblock includes: adjusting a size of the basic encoding block accordingto the block split information; and splitting the boundary block into atleast one encoding block on the basis of the size information of thebasic encoding block, the size information of the boundary block, andinformation regarding the adjusted size of the basic encoding block.

Advantageous Effects

According to the present invention, when the size of an image is not amultiple of the size of a basic encoding block, it is possible toprovide uniformity of encoding by expanding the image such that the sizeof the image is a multiple of the size of the basic encoding block.

According to the present invention, when the size of an image is not amultiple of the size of a basic encoding block, it is possible toprovide high efficiency encoding by adjusting the size of the basicencoding block.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a conceptual view of an image encoding and decoding systemaccording to an embodiment of the present invention.

FIG. 2 is a block diagram of an image encoding apparatus according to anembodiment of the present invention.

FIG. 3 is a block diagram of an image decoding apparatus according to anembodiment of the present invention.

FIG. 4 is a view showing a tree-based block form according to anembodiment of the present invention.

FIG. 5 is a view showing various block forms according to an embodimentof the present invention.

FIG. 6 is a view showing a block for illustrating a block splitaccording to an embodiment of the present invention.

FIG. 7 is a flowchart showing an image encoding method according to anembodiment of the present invention.

FIGS. 8A and 8B are views showing an image partitioned into basicencoding blocks of various sizes according to an embodiment of thepresent invention.

FIG. 9 is a first example diagram illustrating a basic encoding blockadjustment method according to an embodiment of the present invention.

FIG. 10 is a second example diagram illustrating a basic encoding blockadjustment method according to an embodiment of the present invention.

FIGS. 11A and 11B are views showing a boundary block for illustrating abasic encoding block adjustment method according to an embodiment of thepresent invention.

FIG. 12 is a flowchart showing a method of encoding an image byadjusting a basic encoding block according to an embodiment of thepresent invention.

FIGS. 13A and 13B show a first example diagram illustrating an imagesize adjustment method according to an embodiment of the presentinvention.

FIG. 14 is a second example diagram illustrating an image sizeadjustment method according to an embodiment of the present invention.

FIG. 15 is a flowchart showing a method of encoding an image byadjusting the size of the image according to an embodiment of thepresent invention.

FIGS. 16A to 16D show an example diagram for illustrating an adaptiveimage data processing method according to an embodiment of the presentinvention.

FIGS. 17A to 17F show a first example diagram illustrating an adaptiveimage data processing method using a plurality of split methodsaccording to an embodiment of the present invention.

FIGS. 18A to 18C show a second example diagram illustrating an adaptiveimage data processing method using a plurality of split methodsaccording to an embodiment of the present invention.

FIG. 19 is a flowchart showing a method of encoding an image throughadaptive image data processing according to an embodiment of the presentinvention.

DESCRIPTION OF EXAMPLE EMBODIMENTS

While the invention is susceptible to various modifications andalternative forms, specific embodiments thereof are shown by way ofexample in the drawings and will herein be described in detail. Itshould be understood, however, that there is no intent to limit theinvention to the particular forms disclosed, but on the contrary, theinvention is to cover all modifications, equivalents, and alternativesfalling within the spirit and scope of the invention. Like numbers referto like elements throughout the description of the figures.

It will be understood that, although the terms first, second, etc. maybe used herein to describe various elements, these elements should notbe limited by these terms. These terms are only used to distinguish oneelement from another. For example, a first element could be termed asecond element, and, similarly, a second element could be termed a firstelement, without departing from the scope of the present invention. Asused herein, the term “and/or” includes any and all combinations of oneor more of the associated listed items.

It will be understood that when an element is referred to as being“connected” or “coupled” to another element, it can be directlyconnected or coupled to the other element or intervening elements may bepresent. In contrast, when an element is referred to as being “directlyconnected” or “directly coupled” to another element, there are nointervening elements present. Other words used to describe therelationship between elements should be interpreted in a like fashion(i.e., “between” versus “directly between,” “adjacent” versus “directlyadjacent,” etc.).

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting to the invention. Asused herein, the singular forms “a,” “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises,”“comprising,” “includes” and/or “including”, when used herein, specifythe presence of stated features, integers, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, integers, steps, operations, elements,components, and/or groups thereof.

Unless otherwise defined, all terms (including technical and scientificterms) used herein have the same meaning as commonly understood by oneof ordinary skill in the art to which this invention belongs. It will befurther understood that terms, such as those defined in commonly useddictionaries, should be interpreted as having a meaning that isconsistent with their meaning in the context of the relevant art andwill not be interpreted in an idealized or overly formal sense unlessexpressly so defined herein.

Generally, an image may be composed of a series of still images. Thestill images may be classified in units of a group of pictures (GOP),and each still image may be referred to as a picture or a frame. As ahigher concept, units such as a GOP and a sequence may exist, and alsoeach picture may be split into predetermined regions such as slices,tiles, blocks, and the like. Also, one GOP may include units such asPicture I, Picture P, and Picture B. Picture I may refer to a picturethat is autonomously encoded/decoded without using a reference picture,and Picture P and Picture B may refer to a picture that isencoded/decoded by performing a process such as motion estimation andmotion compensation using a reference picture. Generally, Picture P mayuse Picture I and Picture B as reference pictures, and Picture B may usePicture I and Picture P as reference pictures. However, the abovedefinitions may also be changed by settings of encoding/decoding.

Here, a picture referred to in encoding/decoding is called a referencepicture, and a block or pixel referred to in encoding/decoding is calleda reference block or a reference pixel. Also, reference data may includefrequency-domain coefficients and various types of encoding/decodinginformation generated and determined during an encoding/decodingprocess, as well as spatial-domain pixel values.

The minimum unit of the image may be a pixel, and the number of bitsused to represent one pixel is called a bit depth. Generally, the bitdepth may be eight bits, and another bit depth may be supporteddepending on the encoding settings. At least one bit depth may besupported depending on a color space. Also, at least one color space maybe included according to an image color format. One or more pictureshaving the same size or one or more pictures having different sizes maybe included according to a color format. For example, YCbCr 4:2:0 may becomposed of one luminance component (Y in this example) and twochrominance components (Cb/Cr in this example). At this time, the ratioof the luminance component and the chrominance components may be 1:2 inlength and width. As another example, YCbCr 4:4:4 may have the sameratio in length and width. As the above example, when one or more colorspaces are included, a picture may be split into the color spaces.

The present invention will be described on the basis of any color space(Y in this example) of any color format (YCbCr in this example), andthis description will be applied to another color space (Cb and Cr inthis example) of the color format in the same or similar manner(settings dependent on a specific color space). However, a partialdifference (settings independent of a specific color space) may be givento each color space. That is, the settings dependent on each color spacemay refer to settings proportional to or dependent on the componentratio (e.g., 4:2:0, 4:2:2, or 4:4:4), and the setting independent ofeach color space may refer to settings of only a corresponding colorspace, independently from or regardless of the component ratio. In thepresent invention, some elements may have independent settings ordependent settings depending on the encoder/decoder.

Setting information or syntax elements needed during an image encodingprocess may be determined at a level of units such as a video, asequence, a picture, a slice, a tile, a block, and the like. The unitsinclude a video parameter set (VPS), a sequence parameter set (SPS), apicture parameter set (PPS), a slice header, a tile header, and a blockheader. An encoder may add the units to a bitstream and send thebitstream to a decoder. The decoder may parse the bitstream at the samelevel, restore the setting information sent by the encoder, and use thesetting information in an image decoding process. Each parameter set hasa unique ID value, and a lower parameter set may have an ID value of anupper parameter set to be referred to. For example, a lower parameterset may refer to information of an upper parameter set having acorresponding ID value among one or more upper parameter sets. Amongvarious examples of the above-described units, when any one unitincludes one or more different units, the any one unit may be referredto as an upper unit, and the included units may be referred to as alower unit.

Setting information having occurred in such a unit may include settingsindependent of each unit or settings dependent on a previous, following,or upper unit. Here, it will be understood that the dependent settingsindicate setting information of a corresponding unit using flaginformation corresponding to settings of the previous, following, orupper unit (e.g., 1-bit flag; 1 indicates Follow, and 0 indicates Do NotFollow). In the present invention, the setting information will bedescribed focusing on an example of the independent settings. However,an example may also be included in which a relation dependent on thesetting information of the previous, following, or upper unit of thecurrent unit is added to, or substituted for, the independent settings.

Generally, image encoding/decoding may be performed according to aninput size, but may be performed through size adjustment. For example, ahierarchical coding scheme for supporting spatial, temporal, andimage-quality-related scalability may allow adjusting the entireresolution such as image expansion and reduction and also may allowperforming partial image expansion and reduction. Such information maybe switched by assigning selection information in the above-describedunit such as VPS, SPS, PPS, and Slice Header. In this case, a verticalrelation between the units may be set as VPS>SPS>PPS>Slice header.

Preferred embodiments of the present invention will be described belowin more detail with reference to the accompanying drawings.

FIG. 1 is a conceptual view of an image encoding and decoding systemaccording to an embodiment of the present invention.

Referring to FIG. 1 , an image encoding apparatus 105 and an imagedecoding apparatus 100 may be a user terminal such as a personalcomputer (PC), a notebook computer, a personal digital assistant (PDA),a portable multimedia player (PMP), a PlayStation Portable (PSP), awireless communication terminal, a smart phone, and a TV or a serverterminal such as an application server and a service server, and mayinclude a variety of apparatuses having a communication device, such asa communication modem, for communicating with various devices orwired/wireless communication networks, a memory 120 or 125 for storingvarious programs and data for performing inter- or intra-prediction toencode or decode an image, or a processor 110 or 115 for executing theprograms to perform calculation and control, and so on.

In addition, an image encoded into a bitstream by the image encodingapparatus 105 is transmitted to the image decoding apparatus 100 in realtime or in non-real time through a wired/wireless communication networksuch as the Internet, a local area network (LAN), a wireless LAN, aWiBro network, or a mobile communication network and through a varietyof communication interfaces such as a cable, a universal serial bus(USB), or the like. The bitstream is decoded by the image decodingapparatus 100 so that the image may be restored and replayed. Also, theimage encoded into the bitstream by the image encoding apparatus 105 maybe transferred from the image encoding apparatus 105 to the imagedecoding apparatus 100 through a computer readable recording medium.

The above-described image encoding apparatus and decoding apparatus maybe separate apparatuses, but may be provided as a single imageencoding/decoding apparatus according to the implementation. In thiscase, some elements of the image encoding apparatus may be substantiallythe same as those of the image decoding apparatus and may be implementedto include at least the same structures or perform the same functions.

Therefore, in the following detailed description of technical elementsand their working principles, a redundant description of thecorresponding technical elements will be omitted. Also, the imagedecoding apparatus corresponds to a computing apparatus that applies animage encoding method performed by the image encoding apparatus to adecoding process, and thus the following description will focus on theimage encoding apparatus.

The computer apparatus may include a memory configured to store aprogram or a software mode for implementing an image encoding methodand/or an image decoding method and a processor connected to the memoryto execute the program. Here, the image encoding apparatus may also bereferred to as an encoder, and the image decoding apparatus may also bereferred to as a decoder.

FIG. 2 is a block diagram of an image encoding apparatus according to anembodiment of the present invention.

Referring to FIG. 2 , the image encoding apparatus 20 may include aprediction unit 200, a subtractor unit 205, a transformation unit 210, aquantization unit 215, an inverse quantization unit 220, an inversetransformation unit 225, an adder unit 230, a filter unit 235, anencoded picture buffer 240, and an entropy-encoding unit 245.

The prediction unit 200 may be implemented using a prediction module,which is a kind of software module, and may generate a prediction blockto perform intra-prediction or inter-prediction on a block to beencoded. The prediction unit 200 may predict a current block to beencoded in an image to generate a prediction block. In other words, theprediction unit 200 may predict a pixel value of a pixel of a currentblock to be encoded in an image through intra-prediction orinter-prediction to generate a prediction block having a predicted pixelvalue of the pixel. Also, the prediction unit 200 may transferinformation needed to generate the prediction block, such as informationregarding a prediction mode such as an inter-prediction mode or anintra-prediction mode, to an encoding unit so that the encoding unit mayencode the information regarding the prediction mode. In this case, aprocessing unit for performing prediction, a prediction method, and aprocessing unit for determining details may be determined throughencoding/decoding settings. For example, a prediction method, aprediction mode, and the like may be determined by a prediction unit,and prediction may be performed by a transform unit.

An intra-prediction unit may have a directional prediction mode such asa horizontal mode, a vertical mode, and the like used depending on aprediction direction, and a non-directional prediction mode such as DCand Planar using methods such as averaging and interpolation of areference pixel. An intra-prediction mode candidate group may beconfigured through the directional and non-directional modes. Onecandidate may be used, as the candidate group, among various candidatessuch as 35 prediction modes (33 directional modes+2 non-directionalmodes), 67 prediction modes (65 directional modes+2 non-directionalmodes), or 131 prediction modes (129 directional modes+2 non-directionalmodes).

The intra-prediction unit may include a reference pixel configurationunit, a reference pixel filter unit, a reference pixel interpolationunit, a prediction mode determination unit, a prediction blockgeneration unit, and a prediction mode encoding unit. The referencepixel configuration unit may configure, as a reference pixel forintra-prediction, a pixel that belongs to a block adjacent to a currentblock and is placed adjacent to the current block. Depending on encodingsettings, the reference pixel configuration unit may configure the mostadjacent reference pixel line, another adjacent reference pixel line, ora plurality of reference pixel lines as a reference pixel. When some ofthe reference pixels cannot be used, a reference pixel may be generatedusing available reference pixels. When none of the reference pixels canbe used, a reference pixel may be generated using a predetermined value(e.g., a median of a range of pixels values represented by a bit depth).

The reference pixel filter unit of the intra-prediction unit may filterthe reference pixel in order to reduce residual deterioration during anencoding process. In this case, a filter used for the filtering may be alow-pass filter, such as a 3-tap filter [¼, ½, ¼] and a 5-tap filter [2/16, 3/16, 6/16, 3/16, 2/16]. Depending on the encoding information(e.g., a block size, a block form, a prediction mode, etc.), whether toperform filtering and the type of filtering may be determined.

The reference pixel interpolation unit of the intra-prediction unit maygenerate a decimal fraction pixel through a linear interpolation processfor the reference pixel according to the prediction mode. Aninterpolation filter to be applied may be determined according to theencoding information. In this case, the interpolation filter may includea 4-tap Cubic filter, a 4-tap Gaussian filter, a 6-tap Wiener filter, an8-tap Kalman filter, and the like. Generally, the interpolation isperformed separately from the process of performing the low-passfiltering. However, the filters applied to the two processes may beintegrated into a single filter, and the single filter may be used toperform the filtering process.

The prediction mode determination unit of the intra-prediction unit mayselect at least one optimal prediction mode from the prediction modecandidate group in consideration of an encoding cost. The predicationblock generation unit may generate a prediction block using the selectedprediction mode. The prediction mode encoding unit may encode theoptimal prediction mode on the basis of a predicted value. In this case,the prediction information may be adaptively encoded depending onwhether the predicted value is correct or incorrect.

The intra-prediction unit may regard the predicted value as MostProbable Mode (MPM) and may configure some of the modes belonging to theprediction mode candidate group as an MPM candidate group. The MPMcandidate group may include predetermined prediction modes (e.g., DC,Planar, vertical, horizontal, diagonal modes, etc.) or prediction modesof spatially adjacent blocks (e.g., left, upper, upper left, upperright, and lower left blocks). Also, the intra-prediction unit mayconfigure, as the MPM candidate group, a mode induced from the modesincluded in the MPM candidate group.

The prediction modes may have priorities to configure the MPM candidategroup. The order in which the prediction modes are included in the MPMcandidate group may be determined. When the number of prediction modesbelonging to the MPM candidate group (which is determined according tothe number of prediction mode candidate groups) is filled according tothe priorities, the configuration of the MPM candidate group may becompleted. In this case, the priorities may be determined in the orderof the prediction modes of spatially adjacent blocks, the predeterminedprediction modes, and the modes induced from the prediction modesincluded in the MPM candidate group, but may be determined according toother modifications.

For example, the spatially adjacent blocks may be included in thecandidate group in the order of the left block, the upper block, thelower left block, the upper right block, and upper left block. Thepredetermined prediction modes may be included in the candidate group inthe order of DC mode, Planar mode, vertical mode, and horizontal mode.Modes acquired by adding +1, −1, or the like to the pre-included modesmay be included in the candidate group. Thus, a total of six modes mayconstitute the candidate group. Alternatively, a total of seven modesmay constitute the candidate group by sequentially including the leftblock, the upper block, the DC mode, the Planar mode, the lower leftblock, the upper right block, the upper left block, (the left block+1),(the left block−1), and (the upper block+1).

Depending on a motion prediction method performed by theinter-prediction unit, there may be a movement motion model and anon-movement motion model. The movement motion model may performprediction in consideration of only parallel movement, and thenon-movement motion model may perform prediction in consideration ofmotions such as rotation, perspective, and zoom in/out as well asparallel movement. When unidirectional prediction is assumed, themovement motion model may need one motion vector, and the non-movementmotion model may need one or more motion vectors. Each of the motionvectors of the non-movement motion model may be information applied to apredetermined position of the current block, such as an upper leftvertex and an upper right vertex of the current block, and the positionof a region to be predicted in the current block may be acquired througha corresponding motion vector in units of a pixel or a sub-block. Forthe inter-prediction unit, some processes, which will be describedbelow, may be applied in common, and other processes may be appliedindividually.

The inter-prediction unit may include a reference picture configurationunit, a motion estimation unit, a motion compensation unit, a motioninformation determination unit, and a motion information encoding unit.The reference picture configuration unit may add an encoded picturebefore or after the current picture to a reference picture list L0 orL1. A prediction block may be acquired from a reference picture includedin the reference picture list. Depending on the encoding settings, thecurrent image may also be configured as the reference picture and addedto at least one of the reference picture lists.

The reference picture configuration unit of the inter-prediction unitmay include a reference picture interpolation unit and may perform aninterpolation process for a decimal fraction pixel according tointerpolation precision. For example, an 8-tap discrete cosine transform(DCT)-based interpolation filter may be applied to luminance components,and a 4-tap DCT-based interpolation filter may be applied to chrominancecomponents.

The motion estimation unit of the inter-prediction unit performs aprocess of searching for a block having a higher correlation with thecurrent block through the reference picture. In this case, variousmethods such as a full search-based block matching algorithm (FBMA) anda tree step search (TSS) may be used. The motion compensation unitperforms a process of acquiring a prediction block through a motionestimation process.

The motion information determination unit of the inter-prediction unitmay perform a process of selecting optimal motion information of thecurrent block, and the motion information may be encoded by a motioninformation encoding mode such as a skip mode, a merge mode, and acompetition mode. The modes may be configured by combining modessupported depending on the motion models, and may include, for example,a skip mode (movement), a skip mode (non-movement), a merge mode(movement), a merge mode (non-movement), a competition mode (movement),and a competition mode (non-movement). Some of the modes may be includedin the candidate group depending on the encoding settings.

In the motion information encoding mode, a predicted value of the motioninformation (a motion vector, a reference picture, a predicteddirection, etc.) of the current block may be acquired in at least onecandidate bock. When two or more candidate blocks are supported, optimalcandidate selection information may be generated. In the skip mode (withno residual signal) and the merge mode (with a residual signal), thepredicted value may be used as the motion information of the currentblock. In the competition mode, information regarding a differencebetween the predicted value and the motion information of the currentblock may be generated.

A candidate group for predicted values of the motion information of thecurrent block may be adaptive according to the motion informationencoding mode and may have various configurations. The motioninformation regarding blocks (e.g., left, upper, upper left, upperright, lower left blocks, etc.) spatially adjacent to the current blockmay be included in the candidate group, and the motion informationregarding blocks (e.g., left, right, upper, lower, upper left, upperright, lower left, and lower right blocks including a block (at thecenter) in another image corresponding to the current block) temporarilyadjacent to the current block may be included in the candidate group.Hybrid motion information of spatial candidates and temporal candidates(for example, information acquired as an average or a median through themotion block of spatially adjacent blocks and the motion block oftemporarily adjacent blocks; the motion information may be acquired inunits of the current block or a sub-block of the current block) may beincluded in the candidate group.

There may be priorities for configuring a candidate group of thepredicted values of the motion information. The order in which themotion information is included in the predicted-value candidate groupmay be determined. When the number of pieces of the motion informationof the candidate group (which is determined according to the motioninformation encoding mode) is filled according to the priorities, theconfiguration of the candidate group may be completed. In this case, thepriorities may be determined in the order of motion information ofspatially adjacent blocks, motion information of temporarily adjacentinformation, and hybrid motion information of spatial candidates andtemporal candidates, but may be determined according to itsmodification.

For example, the spatially adjacent blocks may be included in thecandidate group in the order of left, upper, upper right, lower left,and upper left blocks, and the temporarily adjacent blocks may beincluded in the order of lower right, middle, right, and lower blocks.

The subtractor unit 205 may subtract the prediction block from thecurrent block to generate a residual block. In other words, thesubtractor unit 205 may calculate a difference between a pixel value ofeach pixel of the current block to be encoded and a predicted pixelvalue of each pixel of the prediction block generated through theprediction unit to generate a residual block, which is a block-typeresidual signal. Also, the subtractor unit 205 may generate the residualblock according to a unit other than a block unit acquired through ablock split unit to be described below.

The transformation unit 210 transforms the residual block into thefrequency domain to transform each pixel value of the residual blockinto a frequency coefficient. Here, the transformation unit 210 maytransform a residual signal into the frequency domain by using varioustransformation techniques for transforming a picture signal on thespatial axis into the frequency axis, such as Hadamard Transform,DCT-Based Transform, Discrete sine transform (DST)-Based Transform, andKarhuhen-Loeve Transform (KTL)-Based Transform. The residual signalconverted in the frequency domain is a frequency coefficient. Thetransformation may be made by a one-dimensional transformation matrix.Each transformation matrix may be adaptively used by a horizontal unitor a vertical unit. For example, for the intra-prediction, when theprediction mode is horizontal, the DCT-based transformation matrix maybe used in a vertical direction, and the DST-based transformation matrixmay be used in a horizontal direction. When the prediction mode isvertical, the DCT-based transformation matrix may be used in ahorizontal direction, and the DST-based transformation matrix may beused in a vertical direction.

The quantization unit 215 quantizes a residual block having a frequencycoefficient obtained by the transformation into the frequency domain bythe transformation unit 210. Here, the quantization unit 215 mayquantize the transformed residual block by using a dead zone uniformthreshold quantization, a quantization weighted matrix, or otherimproved quantization methods. One or more quantization methods may beprovided as candidates and may be determined by an encoding mode,prediction mode information, and the like.

The inverse quantization unit 220 inversely quantizes the residual blockquantized by the quantization unit 215. That is, the inversequantization unit 220 inversely quantizes a quantization frequencycoefficient string to generate a residual block having a frequencycoefficient.

The inverse transformation unit 225 inversely transforms the residualblock that is inversely quantized by the inverse quantization unit 220.That is, the inverse transformation unit 225 inversely transformsfrequency coefficients of the inversely quantized residual block togenerate a residual block having a pixel value, i.e., a restoredresidual block. Here, the inverse transformation unit 225 may performinverse transformation by, in reverse, using the transformation schemeused by the transformation unit 210.

The adder unit 230 adds the residual block restored by the inversetransformation unit 225 to the prediction block predicted by theprediction unit 200 to restore the current block. The restored currentblock may be stored in the encoded picture buffer 240 as a referencepicture (or a reference block) and may be used as the reference picturewhen a block after the current block or another block or picturesubsequent to the current block is encoded.

The filter unit 235 may perform a post-processing filtering process ofone or more of a deblocking filter, a sample adaptive offset (SAO), anadaptive loop filter (ALF), and the like. The deblocking filter mayremove block distortion generated at a boundary between blocks from therestored picture. The ALF may perform filtering on the basis of a valueobtained by comparing an original image to an image restored after theblock is filtered through the deblocking filter. The SAO may restore anoffset difference between the original image and the residual block towhich the deblocking filter is applied, in units of a pixel. Such apost-processing filter may be applied to the restored picture or block.

The encoded picture buffer 240 may store the block or picture restoredthrough the filter unit 235. The restored block or picture stored in theencoded picture buffer 240 may be provided to the prediction unit 200,which is configured to perform intra-prediction or inter-prediction.

The entropy-encoding unit 245 scans a generated quantization frequencycoefficient string according to various scan schemes to generate aquantization coefficient string, encodes the generated quantizationcoefficient string, and outputs the encoded string. A pattern for thescanning may be set as one of various patterns such as a zigzag pattern,a diagonal pattern, a raster pattern, etc. Also, the entropy-encodingunit 245 may generate encoding data including encoding informationdelivered from each element and may output the encoding data in abitstream.

FIG. 3 is a block diagram of an image decoding apparatus according to anembodiment of the present invention.

Referring to FIG. 3 , an image decoding apparatus 30 may include anentropy-decoding unit 305, a prediction unit 310, an inversequantization unit 315, an inverse transformation unit 320, anadder-subtracter 325, a filter 330, and a decoded picture buffer 335.

Also, the prediction unit 310 may include an intra-prediction module andan inter-prediction module.

First, when an image bitstream is received from the image encodingapparatus 20, the received bitstream may be delivered to theentropy-decoding unit 305.

The entropy-decoding unit 305 may decode the bitstream to decodequantized coefficients and decoding data including decoding informationdelivered to each element.

The prediction unit 310 may generate a prediction block on the basis ofthe data delivered from the entropy-decoding unit 305. In this case, areference picture list may be configured using a default configurationtechnique on the basis of a reference image decoded and stored in thedecoded picture buffer 335.

The intra-prediction unit may include a reference picture configurationunit, a reference pixel filter unit, a reference pixel interpolationunit, a prediction block generation unit, and a predictive mode decodingunit. The inter-prediction unit may include a reference pictureconfiguration unit, a motion compensation unit, and a motion informationdecoding unit. Some of the units may perform the same process as theencoder, and some of the units may perform a process for reverseinduction.

The inverse quantization unit 315 may inversely quantize quantizedtransformation coefficients provided in the bitstream and decoded by theentropy-decoding unit 305.

The inverse transformation unit 320 may apply the inverse DCT, theinverse integer transformation, or inverse-transformation techniqueswith similar concepts to the transformation coefficients to generate aresidual block.

In this case, the inverse quantization unit 315 and the inversetransformation unit 320 may be implemented in various methods toinversely perform the process performed by the transformation unit 210and the quantization unit 215 of the image encoding apparatus 20, whichhas been described above. For example, the inverse quantization unit 315and the inverse transformation unit 320 may use inverse transformationand the same process shared with the transformation unit 210 and thequantization unit 215, and the inverse quantization unit 315 mayinversely perform a transformation and quantization process usinginformation (e.g., a transformation size, a transformation form, aquantization type, etc.) regarding the transformation and quantizationprocess received from the image encoding apparatus 20.

A restored image block may be generated by adding the residual block onwhich the inverse quantization and inverse transformation process isperformed to the prediction block derived by the prediction unit 310.The adding may be achieved by the adder-subtracter 325.

A deblocking filter may be applied to the restored image block as thefilter 330 in order to remove a blocking phenomenon if necessary, anddifferent loop filters may be additionally used before and after thedecoding process in order to enhance video quality.

The image block on which the restoration and filtering are performed maybe stored in the decoded picture buffer 335.

Although not shown, the image encoding/decoding apparatus may furtherinclude a picture split unit and a block split unit.

The picture split unit may split (or partition) a picture into at leastone processing unit such as a color space (YCbCr, RGB, or XYZ), a tile,a slice, and a basic encoding unit (or a maximum encoding unit), and theblock split unit may split (or partition) a basic encoding unit into atleast one processing unit (e.g., encoding, prediction, transformation,quantization, entropy, and in-loop filter unit).

The basic encoding unit may be acquired by splitting the picture atcertain distances in horizontal and vertical directions. On the basis ofthe split, the tile, slice, and the like may be split (i.e., a splitunit such as a tile and a slice is configured as an integral multiple ofthe basic encoding block; however, exceptional cases may occur in asplit unit located at an image boundary), but the present invention isnot limited thereto.

For example, a picture may be partitioned into basic encoding units andthen split into the units, or may be partitioned into the units and thensplit into the basic encoding units. The following description assumesthat the partitioning and splitting order of each unit corresponds tothe former. However, the present invention is not limited thereto, andthe latter may be possible according to the encoding/decoding settings.The latter case may be modified into a case in which the size of thebasic encoding unit is adaptive depending on a split unit (a tile, etc.)(i.e., a basic encoding block of a different size may be applied to eachunit).

In the present invention, an example will be described in which a caseof partitioning a picture into basic encoding units is set as a default(i.e., a picture is not split into tiles or slices, or a picture is asingle tile or a single slice). However, it should be understood that,as described above, even when a picture is split into basic encodingunits on the basis of a unit acquired by partitioning each split unitfirst, the settings and the like to be proposed in the following variousexamples are applied without or after being modified.

Among the split units, the slice may be composed of a group of at leastone or more consecutive blocks according to a scanning pattern, and thetile may be composed of a rectangular group of spatially adjacentblocks. Also, any other additional split unit may be supported andconfigured according to its definition. In particular, like a checkeredpattern, a picture may be split into tiles using one or more horizontaland vertical lines. Alternatively, the picture may be composed ofrectangles having various sizes, rather than being uniformly split byhorizontal and vertical lines.

Meanwhile, the picture may be split into encoding units (or blocks)having various sizes through the block split unit. In this case, theencoding unit may be composed of a plurality of encoding blocks (e.g.,one luminance encoding block and two chrominance encoding blocks, etc.)according to a color format, and the size of the blocks may bedetermined according to the color format. For convenience ofdescription, a block corresponding to one color component (luminancecomponent) will be described below as a reference.

It should be understood that the following description is directed toone color component, but may be changed and applied to other colorcomponents in proportion to a length ratio corresponding to the colorformat (e.g., as for YCbCr 4:2:0, a length aspect ratio between theluminance component and the chrominance component is 2:1). Also, itshould be understood that a block split dependent on another colorcomponent (e.g., CbCr is dependent on a result of a block split of Y)may be possible, but a block split independent of each color componentmay be possible. Also, one common block split setting may be used(considering proportion to a length ratio), but it is also necessary tounderstand that an individual block split setting is used depending on acolor component.

The encoding block may have a variable size of M×M (e.g., M is 4, 8, 16,32, 64, 128, or the like). Alternatively, depending on a split scheme(e.g., a tree-based split, a quad tree (QT) split, binary tree (BT),etc.), the encoding block may have a variable size or form of M×N (e.g.,M and N are 4, 8, 16, 32, 64, 128, or the like). In this case, theencoding block may be a unit which is a basis for intra-prediction,inter-prediction, transformation, quantization, entropy coding, and thelike, and it is assumed that the block is a unit that may be acquiredafter each unit is determined.

The block split unit may be set in association with each element of theimage encoding apparatus and the image decoding apparatus. Through thisprocess, the size and form of the block may be determined. In this case,a different block may be defined for each element. The block may be aprediction block for the prediction unit, a transformation block for thetransformation unit, a quantization block for the quantization block, orthe like. However, the present invention is not limited thereto, and anadditional block unit may be defined for another element. The size andform of the block may be defined by horizontal and vertical lengths ofthe block.

A block may be acquired by the block split unit in the range from aminimum value to a maximum value. For example, when the block supportsthe form of a square and has a maximum value of 256×256 and a minimumvalue of 8×8, a block with a size of 2^(m)×2^(m) (here, m is an integerfrom 3 to 8; for example, 8×8, 16×16, 32×32, 64×64, 128×128, and256×256), a block with a size of 2 m×2 m (here, m is an integer from 4to 128), or a block with a size of m×m (here, m is an integer from 8 to128) may be acquired.

Alternatively, when the block supports the form of a square and arectangle and has the same range as described above, a block with a sizeof 2^(m)×2^(n) (here, m and n are integers from 3 to 8; when it isassumed that the maximum aspect ratio is 2:1, 8×8, 8×16, 16×8, 16×16,16×32, 32×16, 32×32, 32×64, 64×32, 64×64, 64×128, 128×64, 128×128,128×256, 256×128, 256×256) may be acquired. However, there may be nolimitation on the aspect ratio depending on the encoding/decodingsettings, or the maximum value of the ratio (e.g., 1:2, 1:3, 1:7, etc.)may exist. Alternatively, a block with a size of 2 m×2n (here, m and nare integers from 4 to 128) may be acquired, and also a block with asize of m×m (here, m and n are integers from 8 to 256) may be acquired.

The block capable of being acquired by the block split unit may bedetermined according to the encoding/decoding settings (e.g., a blocktype, a split scheme, and a split setting). For example, the encodingblock may be acquired as a block with a size of 2^(m)×2^(n), theprediction block may be acquired as a block with a size of 2 m×2n orm×n, and the transformation block may be acquired as a block with a sizeof 2^(m)×2^(n). In other words, information regarding the size and rangeof the blocks (e.g., information related to an exponent and a multiple)may be generated on the basis of the encoding/decoding settings.

The range, such as the maximum value and the minimum value of the blocksize, may be determined depending on the block type. Also, some blocksmay have block range information explicitly generated, and other blocksmay have block range information implicitly generated. For example, theencoding block and the transformation block may have related informationexplicitly generated, and the prediction block may have relatedinformation implicitly processed.

In the explicit case, at least one piece of range information may begenerated. For example, the range information of the encoding block maybe generated as information regarding the maximum value and the minimumvalue and on the basis of a difference between the maximum and thepredetermined minimum value (e.g., eight). In other words, the rangeinformation may be generated based on the settings and may be generatedbased on information regarding an exponent difference value between themaximum value and the minimum value. However, the present invention isnot limited thereto. Also, a plurality of pieces of range informationfor horizontal and vertical lengths of a rectangular block may begenerated.

In the implicit case, the range information may be acquired on the basisof the encoding/decoding settings (e.g., a block type, a split scheme, asplit setting, etc.). For example, for the prediction block, theencoding block (e.g., having a maximum size of M×N and a minimum size ofm×n), which is an upper unit, may acquire information regarding themaximum value and the minimum value according to a candidate group(here, M×N and m/2×n/2) acquirable from split settings (e.g., a quadtree split+a split depth of 0) of the prediction block.

The size and form of an initial block of the block split unit may bedetermined from its upper unit. In other words, the encoding block mayhave the basic encoding block as an initial block, and the predictionblock may have the encoding block as an initial block. Also, thetransformation block may have the encoding block or the prediction blockas an initial block, which may be determined according to theencoding/decoding settings.

For example, the prediction block is an upper unit of the transformationblock when the encoding mode is an intra mode, and the prediction blockis a unit that is independent of the transformation block when theencoding node is an inter mode. The initial block, which is a start unitfor split, may be split into blocks of a small size. When an optimalsize and form corresponding to the block split are determined, the blockmay be determined as an initial block of a lower unit. The initialblock, which is the start unit for the split, may be considered as aninitial block of an upper unit. Here, the upper unit may be an encodingblock, and the lower unit may be a prediction block or a transformationblock. However, the present invention is not limited thereto. Asdescribed above, when the initial block of the lower unit is determined,a split process for finding a block of the optimal size and form may beperformed.

In summary, the block split unit may split the basic encoding unit (orthe maximum encoding unit) into at least one encoding unit (or lowerencoding unit). Also, the encoding unit may be split into at least oneprediction unit and also may be split into at least one transformationunit. The encoding unit may be split into at least one encoding block,and the encoding block may be split into at least one prediction blockand also may be split into at least one transformation block. Also, theprediction unit may be split into at least one prediction block, and thetransformation unit may be split into at least one transformation block.

According to the above example, a different split setting is applieddepending on a block type. Also, some blocks may be combined with otherblocks, and thus a single split process may be performed. For example,when an encoding block and a transformation block are combined into oneunit, a split process for acquiring an optimal block size and form isperformed. Thus, the optimal block size and form may be the optimal sizeand form of the transformation block as well as the optimal size andform of the encoding block. Alternatively, the encoding block and thetransformation block may be combined into one unit, the prediction blockand the transformation block may be combined into one unit, or theencoding block, the prediction block, and the transformation block maybe combined into one unit. Also, the combination of other blocks may bepossible.

When the block of the optimal size and form is found as described above,mode information related to the block (e.g., split information, etc.)may be generated. The mode information may be contained in a bitstreamin addition to information generated in a configuration unit to whichthe block belongs (e.g., prediction-related information andtransformation-related information) and then transmitted to a decoder.The mode information may be parsed by the decoder at units of the samelevel and then used during an image decoding process.

The split scheme will be described below. For convenience ofdescription, it is assumed that the initial block has the form of asquare. However, when the initial block has the form of a rectangle, thesplit scheme may be applied in the same or similar manner. Accordingly,the present invention is not limited thereto.

FIG. 4 is a view showing a tree-based block form according to anembodiment of the present invention.

The block form may be determined according to the type of a tree usedfor splitting. Referring to FIG. 4 , there may be a single 2N×2N blockon which a split is not performed, two 2N×N blocks on which a binarytree-based horizontal split is performed, two N×2N blocks on which abinary tree-based vertical split is performed, four blocks N×N on whicha quad tree-based split is not performed, and the like. However, thepresent invention is not limited thereto.

In detail, acquirable candidate blocks may be blocks 4 a and 4 b whenthe quad tree-based split is performed, and acquirable candidate blocksmay be blocks 4 a, 4 b, and 4 c when the binary tree-based split isperformed.

When the quad tree-based split is performed, one split flag may besupported. Here, the one split flag may be a split presence flag. Inother words, when the split flag of the quad tree is 0, the split is notperformed such that the block 4 a may be acquired. Also, when the splitflag is 1, the split is performed such that the block 4 d may beacquired.

When the binary tree-based split is performed, a plurality of splitflags may be supported. Here, one of the plurality of split flags may bea split presence flag, and another split flag may be a split directionflag. In other words, when the split presence flag, among the splitflags of the binary tree, is 0, the split is not performed such that theblock 4 a may be acquired. Also, when the split presence flag is 1, theblock 4 b or 4 c may be acquired depending on the split direction flag.

FIG. 5 is a view showing various block forms according to an embodimentof the present invention.

Referring to FIG. 5 , a block (with a size of 4N×4N) may be split in thevarious forms depending on split settings and split methods and also maybe split in another form, which is not shown in FIG. 5 .

In an embodiment, a tree-based asymmetric split may be allowed. Forexample, for the binary tree-based split, the block (with a size of4N×4N) may allow symmetric blocks with sizes of 4N×2N (5 b) and 2N×4N (5c) and may allow asymmetric blocks with sizes of 4N×3N/4N×N (5 d),4N×N/4N×3N (5 e), 3N×4N/N×4N (5 f), and N×4N/3N×4N (5 g). When among thesplit flags, a flag for allowing an asymmetric split is explicitly orimplicitly inactivated according to split settings or the like, acandidate block may be the block 5 b or 5 c depending on the splitdirection flag. When a flag for allowing an asymmetric split isactivated, candidate blocks may be the blocks 5 b, 5 d, and 5 e or 5 c,5 f, and 5 g. Here, the blocks 5 d to 5 g according to the asymmetricsplit may have a horizontal or vertical length ratio of 1:3 or 3:1, butthe length ratio may be 1:2, 1:4, 2:3, 2:4, or 3:4. Thus, the presentinvention is not limited thereto.

The split flag of the binary tree may further include a split form flagin addition to the split presence flag and the split direction flag.Here, the split form flag may indicate symmetry or asymmetry. When thesplit form flag indicates asymmetry, a flag indicating a split ratio maybe generated. The flag indicating the split ratio may indicate an indexallocated to a predetermined candidate group. For example, when a splitratio of 1:3 or 3:1 is supported for a candidate group, the split ratiomay be selected through a 1-bit flag.

Also, the split flag of the binary tree may further include the flagindicating the split ratio in addition to the split presence flag andthe split direction flag. In this case, a candidate having a symmetricratio of 1:1 may be included as the candidate for the split ratio.

In the present invention, the binary tree will be described assumingthat configuration is made with an additional split form flag. Unlessotherwise specified, the binary tree may indicate a symmetric binarytree.

In another embodiment, additional tree vision may be allowed to thetree-based split. For example, a ternary tree-based split, a quadtree-based split, an octa tree-based split, and the like may be allowed,and thus n split blocks (here, n is an integer) may be acquired. Inother words, three split blocks may be acquired through the ternarytree, four split blocks may be acquired through the quad type tree, andeight split blocks may be acquired through the octa tree.

The ternary tree may support blocks with sizes of 4N×2N/4N×N_2 (5 h),4N×N/4N×2N/4N×N (5 i), 4N×N_2/4N×2N (5 j), 2N×4N/N×4N_2 (5 k),N×4N/2N×4N/N×4N (5 l), and N×4N_2/2N×4N (5 m), the quad type tree maysupport blocks with sizes of 2N×2N (5 n), 4N×N (5 o), and N×4N (5 p),and the octa tree may support a block with a size of N×N (5 q).

Whether to support a tree-based split may be implicitly determinedaccording to the encoding/decoding settings, and its associatedinformation may be explicitly generated. Also, the binary tree-basedsplit, the ternary tree-based split, and the quad tree-based split maybe used solely or in combination depending on the encoding/decodingsettings.

For example, the binary tree may support the block 5 b or 5 c dependingon the split direction. Accordingly, when it is assumed that the userange of the binary tree partially overlaps that of the ternary tree,the blocks 5 b, 5 c, 5 i, and 5 l may be supported by mixing the binarytree and the ternary tree. When other than the existing flags, a flagfor allowing an additional split is explicitly or implicitly inactivatedaccording to the encoding/decoding settings, an acquirable candidateblock may be a block 5 b or 5 c. When the flag for allowing theadditional split is activated, acquirable candidate blocks may be blocks5 b and 5 i (or 5 b, 5 h, 5 i, and 5 j) or blocks 5 c and 5 l (or 5 c, 5k, 5 l, and 5 m).

Here, a horizontal (left/middle/right) length ratio or a vertical(top/middle/bottom) length ratio of the ternary tree-based split is2:1:1, 1:2:1, or 1:1:2, but other ratios are possible depending on theencoding settings, and thus the present invention is not limitedthereto.

When the ternary tree-based split is performed, a plurality of splitflags may be supported. Here, one of the plurality of split flags may bea split presence flag, another split flag may be a split direction flag,and a split ratio flag may be further included. These split flags may besimilar to those of the binary tree. In the present invention, it isassumed that there is one candidate in which a ratio supported accordingto the split direction is 1:2:1 such that the split ratio flag isomitted.

In the present invention, adaptive encoding/decoding settings may beapplied according to the split scheme.

In an embodiment, the split scheme may be determined according to ablock type. For example, an encoding block and a transformation blockmay be subject to the quad tree-based split, and the prediction blockmay be subject to the quad tree-based split and the binary tree-basedsplit (or the ternary tree-based split).

In another embodiment, the split scheme may be determined according to ablock size. For example, the quad tree-based split may be possible in afirst range (e.g., from a×b to c×d) between the minimum value and themaximum value of the block, and the binary tree-based (or the ternarytree-based) split may be possible in a second range (e.g., from e×f tog×h). Here, the first range may have a larger size than the secondrange, but the present invention is not limited thereto. Here, rangeinformation according to the split scheme may be explicitly generated orimplicitly determined, and the ranges may overlap each other.

In still another embodiment, the split scheme may be determinedaccording to the form of a block (or a block before the split). Forexample, when the block has the form of a square, the quad tree-basedsplit and the binary tree-based (or the ternary tree-based) split may bepossible. Alternatively, when the block has the form of a rectangle, thebinary tree-based (or the ternary tree-based) split may be possible.

The tree-based split may be performed in a recursive way. For example,when a split flag of an encoding block having a split depth of k is 0,the encoding of the encoding block may be performed on the encodingblock having a split depth of k. When the split flag of the encodingblock having a split depth of k is 1, the encoding of the encoding blockmay be performed on four sub-encoding blocks (for the quad tree-basedsplit), two sub-encoding blocks (for the binary tree-based split), orthree sub-encoding blocks (for the ternary tree-based split), each ofwhich has a split depth of k+1. The sub-encoding block may be set as anencoding block k+1 and split into sub encoding blocks k+2. Thishierarchical split scheme may be determined according to split settingssuch as a split range and a split-allowable depth.

In this case, a bitstream structure for presenting split information mayselect one or more scan methods. For example, the bitstream of the splitinformation may be configured on the basis of a split depth order or onthe basis of the present of a split. For example, a method based on asplit depth order refers to a method of acquiring split informationcorresponding to the current level depth and then split informationcorresponding to the next level depth on the basis of an initial blocklevel, and a method based on the presence of a split refers to a methodof preferentially acquiring additional split information of a blocksplit on the basis of the initial block. In this case, anotheradditional scan method may be considered. In the present invention, itis assumed that the bitstream of the split information is configured onthe basis of the presence of a split.

As described above, a split based on a single tree or splits based on aplurality of trees may be supported depending on the encoding/decodingsettings.

When the splits based on the plurality of trees are possible, the splitsmay be performed according to predetermined priorities. In detail, it isassumed that the splits are performed according to the priorities. Whena split with a higher priority is performed (i.e., the split is madeinto two or more blocks), a split at a corresponding split depth ends,and a sub-block is set as an encoding block and then split at the nextsplit depth. When the split with the higher priority is not performed, asplit with the next higher priority may be performed. When none of thetree-based splits are performed, a split process at the correspondingblock ends.

When a split flag corresponding to a higher priority is true (split: O)in the above example, additional split information of the correspondingsplit scheme may follow. When the flag is false (split: X), splitinformation (a split flag, a split direction flag, etc.) of a splitscheme corresponding to the next higher priority may be configured.

When a split is performed by at least one split scheme in the aboveexample, a split may be performed on the next sub-block (or the nextsplit depth) according to the priorities. Alternatively, a splitcorresponding to a higher priority at which a split is not performed asa result of the previous split may be excluded from a split candidategroup of the next sub-block.

For example, when the quad tree-based split and the binary tree-basedsplit are supported and the quad tree has a higher priority and the quadtree-based split is performed at a split depth k, the quad tree-basedsplit may be performed on the sub-block having a split depth k+1 whenthe split is made based on the quad tree.

When the binary tree-based split, instead of the quad tree-based split,is made at the split depth k, the quad tree-based split may be performedon the next sub-block having the split depth k+1 first, or only thebinary tree-based split may be allowed by excluding the quad tree fromthe candidate group. This may be determined according to theencoding/decoding settings. In the present invention, the followingdescription assumes the latter case.

When splits based on a plurality of trees are possible, information forselecting a split scheme may be additionally generated, and the splitmay be performed according to the selected split scheme. In the presentexample, the split information according to the selected split schememay follow.

FIG. 6 is a view showing a block for illustrating a block splitaccording to an embodiment of the present invention.

The size and form of a block acquirable according to one or more splitmethods, starting from a basic encoding block, will be described below,but this is merely an example and the present invention is not limitedthereto. Also, split settings such as a split type, split information,and a split information configuration order to be described below aremerely examples, and the present invention is not limited thereto.

Referring to FIG. 6 , a thick solid line may represent a basic encodingblock, a thick dashed line may represent a quad tree split boundary, anda double solid line may represent a symmetric binary tree splitboundary. Also, a solid line may represent a ternary tree splitboundary, and a thin dashed line may represent an asymmetric binary treesplit boundary.

For convenience of description, it is assumed that an upper left block,an upper right block, a lower left block, and a lower right block (e.g.,64×64) with respect to the basic encoding block (e.g., 128×128) haveindividual block split settings.

Also, it is also assumed that four sub-blocks (upper left, upper right,lower left, and lower right blocks) are acquired from an initial block(e.g., 128×128), and thus the split depth increases from 0 to 1, andsplit settings for the quad tree, i.e., split settings such as themaximum block with a size of 128×128, the minimum block with a size of8×8, and the maximum split depth of 4 are applied to the blocks incommon.

Also, when the splits based on multiple trees are supported by eachsub-block, the size and form of an acquirable block may be determinedthrough split settings for a plurality of blocks. In this example, forthe binary tree and the ternary tree, it is assumed that the maximumblock has a size of 64×64, the minimum block has a length of 4, and themaximum split depth is 4.

Under the assumption, the upper left block, the upper right bock, thelower left block, and the lower right block will be describedindividually.

1. Upper Left Blocks A0 to A6.

An upper left block (with a size of 4 M×4N) represents that a splitbased on a single tree is supported, the size and form of an acquirableblock may be determined through one block split setting such as themaximum block, the minimum block, and the split depth.

The upper left block indicates a case in which one block is acquirableaccording to a split. Split information required for the split operationmay be a split presence flag, and acquirable candidates may be 4 M×4Nand 2 M×2N. Here, the split is not performed when the split presenceflag is 0 and is performed when the split presence flag is 1.

Referring to FIG. 6 , when the split presence flag is 1 while thecurrent depth is 1, the upper left block (with a size of 4 M×4N) mayperform the quad tree-based split. Thus, the depth may increase from 1to 2, and blocks A0 to A3, A4, A5, and A6 (each of which is 2 M×2N) maybe acquired.

When the splits based on multiple trees are supported as describedbelow, information generated when only a quad tree-based split ispossible is composed of a split presence flag as shown in this example,and a description thereof will be omitted.

2. Upper Right Blocks A7 to A11

An upper right block (with a size of 4 M×4N) represents a case in whichsplits based on multiple trees (a quad tree and a binary tree, presenceof priorities for splits, the quad tree→the binary tree) are supported.The size and form of an acquirable block may be determined through splitsettings for a plurality of blocks.

The upper right block represents a case in which a block acquirablethrough a split has a plurality of forms, split information required forthe split operation may be a split presence flag, a split type flag, asplit form flag, and a split direction flag, and acquirable candidatesmay be 4 M×4N, 4 M×2N, 2 M×4N, 4 M×N/4 M×3N, 4 M×3N/4 M×N, M×4N/3 M×4N,and 3 M×4N/M×4N.

When a target block is in a range where both the quad tree-based splitand the binary tree-based split are allowed, different split informationmay be configured depending on whether both, or only one of the quadtree-based split and the binary tree-based split is possible.

In the above description, “possible range” means that a target block(with a size 64×64) may use the quad tree-based split of 128×128 to 8×8and the binary-based split of 64×64 to (64>>m)×(64>>n) (here, m+n is 4),and “possible case” means that when a plurality of split schemes aresupported, both or one of the quad tree-based split and the binarytree-based split is possible depending on a split order, a rule, and thelike.

(1) Case in which Both of the Quad Tree-Based Split and the BinaryTree-Based Split are Possible

TABLE 1 a b c d e QT 1 No Split 0 0 SBT hor 0 1 0 0 ABT hor ¼ 0 1 0 1 0ABT hor ¾ 0 1 0 1 1 SBT ver 0 1 1 0 ABT ver ¼ 0 1 1 1 0 ABT ver ¾ 0 1 11 1

In Table 1, a may represent a flag indicating the presence of the quadtree-based split, and b may represent a flag indicating the presence ofthe binary tree-based split. Also, c may represent a flag indicating asplit direction, d may represent a flag indicating a split type, and emay represent a flag indicating a split ratio.

Referring to Table 1, the split information may indicate that the quadtree (QT)-based split is performed when a is 1 and may further include bwhen a is 0. When b is 0, the split information may indicate that nofurther split is performed in a corresponding block. When b is 1, thesplit information may indicate that the binary tree (BT)-based split isperformed and may further include c and d. The split information mayindicate a horizontal split when c is 0 and a vertical split when cis 1. The split information may indicate that a symmetric binary tree(SBT) split is performed when d is 0, and may indicate am asymmetricbinary tree (ABT) split is performed and may further include e when dis 1. The split information may indicate a split with a horizontal orvertical block ratio of 1:4 when e is 0 and may indicate a split withthe opposite ratio (3:4) when e is 1.

(2) Case in which Only the Binary Tree-Based Split is Possible.

TABLE 2 b c d e No Split 0 SBT hor 1 0 0 ABT hor ¼ 1 0 1 0 ABT hor ¾ 1 01 1 SBT ver 1 1 0 ABT ver ¼ 1 1 1 0 ABT ver ¾ 1 1 1 1

In Table 2, b to e may represent the same flags as those of Table 1.

Referring to Table 2, split information corresponding to a case in whichonly the binary tree-based split is possible may be the same as that inTable 1, except for a flag a indicating the presence of the quadtree-based split. In other words, the split information corresponding tothe case in which only the binary tree-based split is possible may startwith b.

Referring to FIG. 6 , among the upper right blocks, the block A7 is anupper right block before the split and on which the binary tree-basedsplit is performed although the quad tree-based split is allowed.Accordingly, the split information may be generated according to Table1.

On the other hand, the blocks A8 to A11 are an upper right block beforethe split and on which the binary tree-based split is performed ratherthan the quad tree-based split, that is, the quad tree-based split is nolonger possible. Accordingly, the split information may be generatedaccording to Table 2.

3. Lower Left Blocks A12 to A15

A lower left block (with a size of 4 M×4N) represents a case in whichsplits based on multiple trees (a quad tree, a binary tree, and aternary tree, presence of priorities for splits, the quad tree→thebinary tree or the ternary tee; also, split scheme selection ispossible; the binary tree or the ternary tree) are supported. The sizeand form of an acquirable block may be determined through split settingsfor a plurality of blocks.

The lower left block represents a case in which a block acquirablethrough a split has a plurality of sizes and forms, split informationrequired for the split operation may be a split presence flag, a splittype flag, and a split direction flag, and acquirable candidates may be4 M×4N, 4 M×2N, 2 M×4N, 4 M×N/4 M×2N/4 M×N, and M×4N/2 M×4N/M×4N.

When a target block is in a range where all of the quad tree-basedsplit, the binary tree-based split, and the ternary tree-based split areallowed, different split information may be configured depending onwhether all of the quad tree-based split, the binary tree-based split,and the ternary tree-based split are possible or not.

(1) Case in which Both of the Quad Tree-Based Split and the BinaryTree-Based/Ternary Tree-Based Split are Possible

TABLE 3 a b c d QT 1 No Split 0 0 BT hor 0 1 0 0 TT hor 0 1 0 1 Bt ver 01 1 0 TT ver 0 1 1 1

In Table 3, a may represent a flag indicating the presence of the quadtree-based split, and b may represent a flag indicating the presence ofthe binary tree-based split or the ternary tree-based split. Also, c mayrepresent a flag indicating a split direction, and d may represent aflag indicating a split type.

Referring to Table 3, the split information may indicate that the quadtree (QT)-based split is performed when a is 1 and may further include bwhen a is 0. When b is 0, the split information may indicate that nofurther split is performed in a corresponding block. When b is 1, thesplit information may indicate that the binary tree (BT)-based or theternary tree (TT)-based split is performed and may further include c andd. The split information may indicate a horizontal split when c is 0 anda vertical split when c is 1. The split information may indicate thatthe binary tree (BT)-based split is performed when d is 0, and mayindicate that the ternary tree (TT)-based split is performed when d is1.

(2) Case in which Only the Binary Tree-Based/Ternary Tree-Based Split isPossible.

TABLE 4 b c d No Split 0 BT hor 1 0 0 TT hor 1 0 1 Bt ver 1 1 0 TT ver 11 1

In Table 4, b to d may represent the same flags as those of Table 3.

Referring to Table 4, split information corresponding to a case in whichonly the binary tree-based/ternary tree-based split is possible may bethe same as that in Table 3, except for a flag a indicating the presenceof the quad tree-based split. In other words, the split informationcorresponding to the case in which only the binary tree-based/ternarytree-based split is possible may start with b.

Referring to FIG. 6 , among the lower left blocks, the blocks A12 andA15 are lower left blocks before the split, and on which the binarytree-based/ternary tree-based split is performed although the quadtree-based split is allowed. Accordingly, the split information may begenerated according to Table 3.

On the other hand, the blocks A13 and A14 are lower left blocks beforethe split and on which the ternary tree-based split is performed ratherthan the quad tree-based split.

Accordingly, the split information may be generated according to Table4.

4. Lower Right Blocks A16 to A20

A lower right block (with a size of 4 M×4N) represents a case in whichsplits based on multiple trees (which is the same as the lower leftblocks; however, in this example, an asymmetric binary tree issupported) are supported. The size and form of an acquirable block maybe determined through split settings for a plurality of blocks.

The lower right block represents a case in which a block acquirablethrough a split has a plurality of forms, split information required forthe split operation may be a split presence flag, a split type flag, asplit form flag, and a split direction flag, and acquirable candidatesmay be 4 M×4N, 4 M×2N, 2 M×4N, 4 M×N/4 M×3N, 4 M×3N/4 M×N, M×4N/3 M×4N,3 M×4N/M×4N, 4 M×N/4 M×2N/4 M×N, and M×4N/2 M×4N/M×4N.

When a target block is in a range where all of the quad tree-basedsplit, the binary tree-based split, and the ternary tree-based split areallowed, different split information may be configured depending onwhether all of the quad tree-based split, the binary tree-based split,and the ternary tree-based split are possible or not.

(1) Case in which Both of the Quad Tree-Based Split and the BinaryTree-Based/Ternary Tree-Based Split are Possible

TABLE 5 a b C d e f QT 1 No Split 0 0 TT hor 0 1 0 0 SBT hor 0 1 0 1 0ABT hor ¼ 0 1 0 1 1 0 ABT hor ¾ 0 1 0 1 1 1 TT ver 0 1 1 0 SBT ver 0 1 11 0 ABT ver ¼ 0 1 1 1 1 0 ABT ver ¾ 0 1 1 1 1 1

In Table 5, a may represent a flag indicating the presence of the quadtree-based split, and b may represent a flag indicating the presence ofthe binary tree-based split or the ternary tree-based split. Also, c mayrepresent a flag indicating a split direction, and d may represent aflag indicating a split type. e may represent a flag indicating a splitform, and f may represent a flag indicating a split ratio.

Referring to Table 5, the split information may indicate that the quadtree (QT)-based split is performed when a is 1 and may further include bwhen a is 0. When b is 0, the split information may indicate that nofurther split is performed in a corresponding block. When b is 1, thesplit information may indicate that the binary tree (BT)-based or theternary tree (TT)-based split is performed and may further include c andd. The split information may indicate a horizontal split when c is 0 anda vertical split when c is 1. When d is 0, the split information mayindicate that the ternary tree (TT)-based split is performed. When d is1, the split information may indicate that the binary tree (BT)-basedsplit is performed and may further include e. When e is 0, the splitinformation may indicate that the symmetric binary tree (SBT)-basedsplit is performed. When e is 1, the split information may indicate thatthe asymmetric binary tree (ABT)-based split is performed and mayfurther include f. The split information may indicate a split with ahorizontal or vertical block ratio of 1:4 when f is 0 and may indicate asplit with the opposite ratio (3:4) when f is 1.

(2) Case in which Only the Binary Tree-Based or the Ternary Tree-BasedSplit is Possible.

TABLE 6 b C d e f No Split 0 TT hor 1 0 0 SBT hor 1 0 1 0 ABT hor ¼ 1 01 1 0 ABT hor ¾ 1 0 1 1 1 TT ver 1 1 0 SBT ver 1 1 1 0 ABT ver ¼ 1 1 1 10 ABT ver ¾ 1 1 1 1 1

In Table 6, b to f may represent the same flags as those of Table 5.

Referring to Table 6, split information corresponding to a case in whichonly the binary tree-based/ternary tree-based split is possible may bethe same as that in Table 5, except for a flag a indicating the presenceof the quad tree-based split. In other words, the split informationcorresponding to the case in which only the binary tree-based/ternarytree-based split is possible may start with b.

Referring to FIG. 6 , among the lower right blocks, the block A20 is alower left block before the split and on which the binarytree-based/ternary tree-based split is performed although the quadtree-based split is allowed. Accordingly, the split information may begenerated according to Table 5.

On the other hand, the blocks A16 and A19 are lower right blocks beforethe split and on which the binary tree-based split is performed ratherthan the quad tree-based split. Accordingly, the split information maybe generated according to Table 6.

FIG. 7 is a flowchart showing an image encoding method according to anembodiment of the present invention.

Referring to FIG. 7 , an image encoding apparatus according to anembodiment of the present invention may set size information of apicture and size information of a basic encoding block (S710) andpartition the picture into basic encoding blocks (S720). Also, the imageencoding apparatus may encode the basic encoding block in a block scanorder (S730) and transmit a bitstream corresponding to encoding to animage decoding apparatus.

Also, the image decoding apparatus according to an embodiment of thepresent invention may operate corresponding to the above-described imageencoding apparatus. In other words, the image decoding apparatus mayrestore the size information of the picture and the size information ofthe basic encoding block on the basis of the bitstream received from theimage encoding apparatus and may partition the picture into basicencoding blocks. Also, the image decoding apparatus may decode the basicencoding block in the block scan order.

Here, the size information of the basic encoding block may representsize information of the maximum block to be encoded, and the block scanorder may be a raster scan order, but may follow various scan orders.Accordingly, the present invention is not limited thereto.

The size information may be expressed as an exponent of 2, the sizeinformation of the maximum block may be expressed as an exponent of 2 oras a difference in exponent of 2 from a predetermined specific size(e.g., 8 or 16). For example, when the basic encoding block has a valueranging from 8×8 to 128×128, the size information may be generated as adifference in exponent from 8. When the maximum block is a size of64×64, the size information may be generated as log₂(64)−log₂(8)=3. Inthis example, the size information of the basic encoding block has beendescribed, but the description may be applied even when themaximum/minimum size information is explicitly generated depending on atree type in the block split settings.

However, when the size of the picture is not configured as an integralmultiple of the basic encoding block, encoding/decoding cannot beperformed by the encoding/decoding as described above, and thus variousmodifications may be required. This will be described in detail below.

Also, in the following example, it is assumed that the picture iscomposed of one slice or tile. However, even when the picture is splitinto a plurality of slices or tiles, the block may not be composed of abasic encoding block size on a right or lower boundary with respect tothe picture. This may be possible under settings in which a picture issplit into blocks and then each of the blocks is split into tiles orslices (i.e., a unit located on the boundary may not be configured as anintegral multiple of the block).

Also, when the picture is split into tiles, slices, or the like and theninto blocks, other cases may be present. That is, since the size of thetiles, slices, or the like is set first, each of the tiles and theslices may not be configured as an integral multiple of the basicencoding block. This means that a block on the middle of the picture, aswell as the block on the right or lower boundary with respect to thepicture, may not be composed of a basic encoding block size. Thus, thismay be applied to the following example, but may be applied inconsideration of the fact that each of the tiles and the slices is notconfigured as an integral multiple of the block.

FIGS. 8A and 8B are views showing an image partitioned into basicencoding blocks of various sizes according to an embodiment of thepresent invention.

FIG. 8A is a view in which a 832×576 image is partitioned into 64×64basic encoding blocks. Referring to FIG. 8A, the horizontal length ofthe picture (image) may be thirteen times as great as, and be anintegral multiple of, the horizontal length of the basic encoding block,and the vertical length of the picture may be nine times as great as,and be an integral multiple of, the vertical length of the basicencoding block. Thus, the image may be partitioned into 117 basicencoding blocks and then encoded/decoded.

FIG. 8B is a view in which a 832×576 image is partitioned into 128×128basic encoding blocks. Referring to FIG. 8B, the horizontal length ofthe picture is 6.5 times as great as, and is not an integral multipleof, the horizontal length of the basic encoding block, and the verticallength of the picture is 4.5 times as great as, and is not an integralmultiple of, the vertical length of the basic encoding block.

In the present invention, an outer boundary of the image may refer to aboundary to the left, to the right, above, below, at an upper leftcorner, at a lower left corner, at an upper right corner, and at a lowerright corner. However, generally, under an assumption that a block splitis performed with respect to upper left coordinates of the image, thefollowing description will focus on the boundary to the right, below,and at the lower right of the picture.

Referring to FIG. 8B, when a region located on the outer boundary of theimage is partitioned into basic encoding blocks, coordinates in an upperleft corner of a corresponding block may be present inside the picture,but coordinates in a lower right corner may be present outside thepicture. Depending on the location of the corresponding block (a rightboundary, a lower boundary, and an upper right boundary), coordinates inan upper right corner may be present outside the picture (P_R),coordinates in a lower left corner may be present outside the picture(P_D), and both of the coordinates may be present outside the picture(P_DR).

Here, 24 128×128 basic encoding blocks, which are not located on theouter boundary of the image may be encoded/decoded, but each of theregions P_R, P_D, and P_DR located on the outer boundary of the image isnot partitioned into basic encoding blocks. Thus, other processing maybe required.

In order to solve such a situation, various methods may be supported.For example, a method of expanding (or padding) and encoding/decoding animage, a method of specifying an exceptional case of basic encodingblocks to perform encoding/decoding, and the like may be supported.Here, the method of expanding and encoding/decoding the image may beperformed in order to maintain uniformity of image encoding/decodingstructure, and the method of specifying the exceptional case of thebasis encoding blocks may be performed in order to enhance imageencoding/decoding performance.

In the present invention, in order to solve a case in which the size ofthe picture is not an integral multiple of the size of the basicencoding block, 1: a basic encoding block adjustment method, 2: an imagesize adjustment method, 3: the adaptive image data processing method,and 4: an adaptive image data processing method according to a pluralityof split methods will be described in detail.

1. Basic Encoding Block Adjustment Method

The basic encoding block adjustment method is a method of borrowing atleast one block supported by the block split unit when the size of thepicture is not an integral multiple of the size of the basic encodingblock.

FIG. 9 is a first example diagram illustrating a basic encoding blockadjustment method according to an embodiment of the present invention.

Referring to FIG. 9 , when the picture has a size of 1856×928, the basicencoding block has a size of 128×128, a split based on a single treesuch as a quad tree is supported, and the minimum block has a size of8×8, the basic encoding block may borrow 64×64, 32×32, 16×16, and 8×8blocks through the block split unit. Here, when the 128×128 block havinga split depth of 0 is not split, the 128×128 block may be borrowed, butthe size of the picture is not an integral multiple of 128×128. Thus,the 128×128 block may be removed from the candidate group.

Since the picture has a horizontal length of 1856 and the basic encodingblock has a horizontal length of 128, a length of 64 may be left on aright boundary of the picture. Here, since the block located on theright boundary of the picture may have a size of 64×128, the basicencoding block may borrow a 64×64 block among the candidate blockssupported by the block split unit. Here, the borrowed block may be amaximum-sized block or a minimum number of blocks with respect to 64×128among the candidate blocks. When a 64×64 block is borrowed, 64×128 maybe composed of two 64×64 blocks.

Also, since the picture has a vertical length of 928 and the basicencoding block has a vertical length of 128, a length of 32 may be lefton a lower boundary of the picture. Here, since the block located on thelower boundary of the picture may have a size of 128×32, the basicencoding block may borrow a 32×32 block among the candidate blockssupported by the block split unit. Here, the borrowed block may be amaximum-sized block or a minimum number of blocks with respect to 128×32among the candidate blocks. When a 32×32 block is borrowed, 128×32 maybe composed of four 32×32 blocks.

Since the block located on the lower right boundary of the picture has asize of 64×32, the basic encoding block may borrow a 32×32 block amongthe candidate blocks supported by the block split unit. Here, theborrowed block may be a block having a maximum size or a minimum numberof blocks with respect to 64×32 among the candidate blocks. When the32×32 block is borrowed, 64×32 may be composed of two 32×32 blocks.

In other words, one basic encoding block (with a size of 128×128) may becomposed of two adjusted basic encoding blocks (with a size of 64×64) onthe right boundary of the image, four adjusted basic encoding blocks(with a size of 32×32) on the lower boundary of the image, or twoadjusted basic encoding blocks (with a size of 32×32) on the lower rightboundary of the image.

FIG. 10 is a second example diagram illustrating a basic encoding blockadjustment method according to an embodiment of the present invention.

Referring to FIG. 10 , when the size of the picture and the size of thebasic encoding block are the same as those shown in FIG. 9 , splitsbased on multiple trees such as a quad tree and a boundary tree aresupported, and the minimum block is 8×8, the basic encoding block mayborrow a square block such as 64×64, 32×32, 16×16, and 8×8 and arectangular block such as 128×64, 64×128, 64×32, 32×64, 32×16, 16×32,16×8, and 8×16 through the block split unit. Here, even a block withhorizontal and vertical ratios exceeding two times may be borrowedthrough the quad tree and the binary tree. However, for convenience ofdescription, the block may be excluded from the candidate group, but thepresent invention is not limited thereto.

Since the picture has a horizontal length of 1856 and the basic encodingblock has a horizontal length of 128, a length of 64 may be left on aright boundary of the picture. Here, since the block located on theright boundary of the picture may have a size of 64×128, the basicencoding block may borrow a 64×128 block among the candidate blockssupported by the block split unit. Here, the borrowed block may be amaximum-sized block or a minimum number of blocks with respect to 64×128among the candidate blocks. When a 64×128 block is borrowed, 64×128 maybe composed of one 64×128 block.

Also, since the picture has a vertical length of 928 and the basicencoding block has a vertical length of 128, a length of 32 may be lefton a lower boundary of the picture. Here, since the block located on thelower boundary of the picture may have a size of 128×32, the basicencoding block may borrow a 64×32 block among the candidate blockssupported by the block split unit. Here, the borrowed block may be amaximum-sized block or a minimum number of blocks with respect to 128×32among the candidate blocks. When the 64×32 block is borrowed, 128×32 maybe composed of two 64×32 blocks.

Since the block located on the lower right boundary of the picture has asize of 64×32, the basic encoding block may borrow a 64×32 block amongthe candidate blocks supported by the block split unit. Here, theborrowed block may be a maximum-sized block or a minimum number ofblocks with respect to 64×32 among the candidate blocks. When the 64×32block is borrowed, 64×32 may be composed of one 64×32 block.

In other words, one basic encoding block (with a size of 128×128) may becomposed of one adjusted basic encoding block (with a size of 64×128) onthe right boundary of the image, two adjusted basic encoding blocks64×32 on the lower boundary of the image, or two adjusted basic encodingblocks 64×32 on the lower right boundary of the image.

Referring to FIG. 10 , the borrowed block may be determined (byperforming a primary process) while the horizontal or vertical length ofthe block is maintained appropriately for the remaining horizontal orvertical length. In other words, on the right side of the picture, a64×128 block is determined to be borrowed among 64×128, 64×64, and 64×32blocks, which are represented using the remaining horizontal length of64. On the lower side of the picture, a 64×32 block is determined to beborrowed among 64×32, 32×32, and 16×32 blocks, which are representedusing the remaining vertical length of 32.

However, the borrowed block may be determined by performing a secondaryor higher process when a block appropriate for the remaining horizontalor vertical length is not present among the blocks supported by theblock split unit. This will be described in detail below with referenceto FIG. 10 .

FIGS. 11A and 11B are views showing a boundary block for illustrating abasic encoding block adjustment method according to an embodiment of thepresent invention.

Referring to FIG. 11A, it is assumed that a 128×80 block is present on alower boundary of the picture because the basic encoding block has asize of 128×128 and the size of the picture is not an integral mixtureof the size of the basic encoding block. Here, the 128×80 block may bereferred to as a boundary block.

The basic encoding block may borrow a 128×64 block, which is one of the128×64, 64×64, and 32×64 blocks and is similar to the size of 128×80,from among the candidate blocks supported by the block split unit, and a128×16 block may be left.

Here, the basic encoding block may additionally borrow a 32×16 block,which is one of the 32×16, 16×16, and 8×16 blocks and is similar to thesize of 128×16, from among the candidate blocks supported by the blocksplit unit.

Referring to FIG. 11B, it is assumed that a 56×128 block is present on aright boundary of the picture because the basic encoding block has asize of 128×128 and the size of the picture is not an integral mixtureof the size of the basic encoding block.

The basic encoding block may borrow a 32×64 block, which is one of the32×64, 32×32, and 32×16 blocks and is similar to the size of 56×128,from among the candidate blocks supported by the block split unit, and a24×128 block may be left.

Here, the basic encoding block may additionally borrow a 16×32 block,which is one of the 16×32, 16×16, and 16×8 blocks and is similar to thesize of 24×128, from among the candidate blocks supported by the blocksplit unit, and an 8×128 block may be left.

Last, the basic encoding block may borrow an 8×16 block, which issimilar to the size of 8×128, from among the candidate blocks supportedby the block split unit.

In other words, FIG. 11A shows an example in which one basic encodingblock (with a size of 128×128) may be composed of one basic encodingblock (with a size of 128×64) that is first adjusted and four basicencoding blocks (with a size of 32×16) that are second adjusted, andFIG. 11B shows an example in which one basic encoding block (with a sizeof 128×128) may be composed of two basic encoding blocks (with a size of32×64) that are first adjusted, four basic encoding blocks (with a sizeof 16×32) that are second adjusted, and eight basic encoding blocks(with a size of 8×16) that are third adjusted.

FIG. 12 is a flowchart showing a method of encoding an image byadjusting a basic encoding block according to an embodiment of thepresent invention.

Referring to FIG. 12 , the image encoding apparatus for adjusting abasic encoding block according to an embodiment of the present inventionmay set size information of a picture and size information and blocksplit information of a basic encoding block (S1210) and may partitionthe picture into basic encoding blocks or basic encoding blocks resetbased on the block split information depending on a block location(S1220). Also, the image encoding apparatus may encode the basicencoding block in a reset block scan order (S1230) and transmit abitstream corresponding to encoding to an image decoding apparatus.

Also, the image decoding apparatus according to an embodiment of thepresent invention may operate corresponding to the above-described imageencoding apparatus. In other words, the image decoding apparatus mayrestore the size information of the picture and the size information andthe block split information of the basic encoding block on the basis ofthe bitstream received from the image encoding apparatus and maypartition the picture into basic encoding blocks or basic encodingblocks reset based on the block split information. Also, the imagedecoding apparatus may decode the basic encoding block in the resetblock scan order.

Here, the block split information may refer to split informationregarding a block acquirable between a maximum block size and a minimumblock size. In detail, the block split information may refer toinformation (e.g., tree information, tree-specific maximum and minimumsize information, maximum split depth information, and a block formlimitation information <horizontal/vertical ratio information>, etc.,which are supported by a color component <Y/Cb/Cr>, an image type<I/P/B>, and the like) that is explicitly generated in association withblock split settings capable of affecting acquisition of blocks havingvarious sizes and forms through the block acquisition unit and mayinclude implicit information such as a split order, a split rule, andthe like that are predetermined by the encoder/decoder.

Also, when the picture is not an integral multiple of the basis encodingblock, the picture is not partitioned into the basic encoding blocks ona right or lower boundary of the picture. Thus, the partitioning may beperformed after a basic encoding block having a block size supportedaccording to the block split settings supported by the block split unitis reset. In other words, one basic encoding block located on a boundaryof the picture may be composed of at least one reset basic encodingblock, and then the partitioning may be performed.

Also, since the boundary of the picture may be composed of one or morebasic encoding blocks, some scan orders such as z-scan may be applied tothe basic encoding block located on the boundary of the picture. Forexample, in the scan order shown in FIG. 11A, a 128×64 block may bescanned first, and then four 32×16 blocks located at a lower side may bescanned from left to right.

2. Image Size Adjustment Method

An image size adjustment method is a method of expanding the size of animage when the size of a picture is not an integral multiple of the sizeof the basic encoding block.

FIGS. 13A and 13B show a first example diagram illustrating an imagesize adjustment method according to an embodiment of the presentinvention.

Referring to FIG. 13A, when the picture has a size of 832×576 and thebasic encoding block has a size of 128×128, a length of 64 may be lefton the right boundary and the lower boundary of the picture. In thiscase, the size of the picture may be horizontally and verticallyexpanded or adjusted by a length of 64.

Referring to FIG. 13 b , the expanded size of the picture may have asize of 896×640, and the expanded picture may be partitioned into 35128×128 basic encoding blocks.

The image size adjustment method will be described in more detail. Thepicture expansion may be performed such that the expanded size of thepicture is the minimum integral multiple of the basic encoding block.Referring to FIG. 11A, since the horizontal length of the picture is832, a length that is greater than the horizontal length of the pictureand that is an integral multiple of the basic encoding block may include896, 1024, and the like. Among the lengths, the minimum length is 986,and thus the horizontal length of the picture may be expanded to 896.Also, since the vertical length of the picture is 576, a length that isgreater than the vertical length of the picture and that is an integralmultiple of the basic encoding block may include 640, 768, and the like.Among the lengths, the minimum length is 640, and thus the verticallength of the picture may be expanded to 640. However, a criterion forthe picture expansion may be differently set, and thus a descriptionthereof will be omitted.

Here, an expanded region may be filled with a predetermined pixel valueor with a pixel value located on a boundary of the expanded region.

In an embodiment, when the expanded region is filled with apredetermined pixel value, the predetermined pixel value may be onevalue included in a pixel value range determined by a bit depth or onevalue included in a range of pixel values constituting an actual image.

Here, the pixel value range determined by the bit depth may be from theminimum value of 0 to the maximum value 2n−1 (here, n is the number ofbits). For example, when a median is used as the predetermined pixelvalue in the pixel value range determined by the bit depth and thenumber of bits is 10, the predetermined pixel value may be 512 in therange from 0 to 1023. Also, in a range of pixel values constituting anactual image, a median may be used as the predetermined pixel value. Forexample, when the range of pixel values constituting an actual imagerange from the minimum value x to the maximum value y, a median in therange from x to y may be used as the predetermined pixel value. A methodof selecting the above-described predetermined pixel value is oneexample, and thus the present invention is not limited thereto.

In another embodiment, when the expanded region is filled with the pixelvalues located on the boundary of the region, pixels located on a rightboundary and a lower boundary of the image may be used. In other words,a region expanded from the right boundary of the image may be filledwith pixels obtained by horizontally copying the pixels located on theright boundary of the image, and a region expanded from the lowerboundary of the image may be filled with pixels obtained by verticallycopying the pixels located on the lower boundary of the image. A regionexpanded from the lower right boundary of the image may be filled withpixels obtained by diagonally copying the pixels located on the lowerright boundary of the image.

Also, for some images, such as a 360 degree image, data of the expandedregion may be acquired from a region having continuity (or correlation)in a three-dimensional (3D) space (e.g., a sphere). For example, when aregion expanded to the right of the image is filled in a protectionformat such as Equirectangular Projection, the expanded region may befilled with pixel values located to the left of the image. Due tosettings for placing a 3D image in a two-dimensional (2D) space, this islocated on an image boundary (i.e., in the 2D space), but may actuallybe an example in which data with correlation is present in the image.

The data filling method may correspond to a method of expanding animage, except for a 360 degree image, by filling an expanded region withunnecessary data and then performing encoding/decoding.

FIG. 14 is a second example diagram illustrating an image sizeadjustment method according to an embodiment of the present invention.

The expanded region of the image may be determined depending on the sizeof the image and the size of the basic encoding block, and a ratio ofthe expanded region to the basic encoding block located on the boundaryof the image is smaller than or larger than a ratio of a regionincluding an actual image to the basic encoding block.

Referring to FIG. 14 , when the size of the basic encoding block is64×64 and the size of the picture is not an integral multiple of thesize of the basic encoding block, the picture may be expanded from theright boundary by a horizontal length of 10 and from the lower boundaryby a horizontal length of 52. In other words, the region expanded fromthe right boundary may be smaller than a region of the existing imagelocated in the basic encoding block, and the region expanded from thelower boundary may be larger than a region of the existing image locatedin the basic encoding block.

Thus, the basic encoding block including the right boundary of thepicture has a relative small expanded region and thus may have a smallreduction in encoding/decoding performance while the basic encodingblock including the lower boundary of the picture has a relative largeexpanded region and thus may have a large reduction in encoding/decodingperformance.

In order to solve this problem, a method of borrowing one of the blockssupported by the block split unit or a method of minimizing expandeddata may be used. The method of borrowing one of the blocks supported bythe block split unit has been described above. Therefore, the method ofminimizing the expanded data will be described below.

In an embodiment, it is assumed that a split based on a single tree suchas a quad tree may be supported for the basic encoding block, theminimum block may be 8×8, and 64×64, 32×32, 16×16, and 8×8 square blocksmay be borrowed through the block split unit. It is also assumed thatthe boundary portion of the picture may be extended in accordance withone of the blocks according to the encoding/decoding settings and thusthe basic encoding block may be (re)set or (re)adjusted. Here, theencoding/decoding settings may be the same as those of a minimum numberof blocks, a block having a smaller difference in size, and the likeamong blocks with sizes greater than the left size. However, the presentinvention is not limited thereto.

Referring to FIG. 14 , a length of 54 may be left on the right boundaryof the picture, and a block located on the right boundary may have asize of 54×64. Here, the block located on the right boundary may beexpanded from the right boundary by a length of 10 in order to borrow a64×64 block, which has a small difference in size from the block locatedon the right boundary, from among the candidate blocks capable of beingsupported by the block split unit. Accordingly, the expanded region of64×64 may be composed of one adjusted basic encoding 64×64 block, butthe size of the adjusted basic encoding block is the same of the basicencoding block before the adjustment. Accordingly, this case may be thesame as a case in which the size of the picture is expanded inaccordance with an integral multiple of the size of the basic encodingblock.

A length of 12 may be left on the lower boundary of the picture, and ablock located on the lower boundary may have a size of 64×12. Here, theblock located on the lower boundary may be expanded from the lowerboundary by a length of 4 in order to borrow a 16×16 block, which has asmall difference in size from the block located on the lower boundary,from among the candidate blocks capable of being supported by the blocksplit unit. Accordingly, the expanded region (with a size of 64×16) maybe composed of four adjusted basic encoding blocks (with a size of16×16).

In another embodiment, it is assumed that splits based on multiple treessuch as a quad tree and a binary tree are supported for the basicencoding block and rectangular blocks such as 64×32, 32×64, 64×16,16×64, 64×8, 8×64, 32×16, 16×32, 32×8, 8×32, 16×8, and 8×16 as well as64×64, 32×32, 16×16, and 8×8 square blocks may be borrowed through theblock split unit.

A length of 54 may be left on the right boundary of the picture, and ablock located on the right boundary may have a size of 54×64. Here, theblock located on the right boundary may be expanded from the rightboundary by a length of 10 in order to borrow a 64×64 block, which hasthe minimum difference, from among candidate blocks having a horizontallength of 64 (64×64, 64×32, 64×16, and 64×8 blocks), which has a smalldifference from the horizontal length of the block located on the rightboundary.

A length of 12 may be left on the lower boundary of the picture, and ablock located on the lower boundary may have a size of 64×12. Here, theblock located on the lower boundary may be expanded from the lowerboundary by a length of 4 in order to borrow a 64×16 block, which hasthe minimum difference, from among candidate blocks having a verticallength of 16 (64×16, 32×16, 16×16, and 8×16 blocks), which has a smalldifference in size from the block located on the right boundary.Accordingly, the expanded region (with a size of 64×16) may be composedof one adjusted basic encoding block (with a size of 64×16).

In other words, the picture may be expanded in order to borrow acandidate block with a small difference in size on the basis of aremaining length according to a difference between the integral multipleof the basic encoding block and the size of the picture, and the basicencoding block may be (re)set or (re)adjusted depending on the size ofthe candidate block having a difference in size.

The image size adjustment method may be similar to the basic encodingblock adjustment method in that the basic encoding block can beadjusted, but is different from the basic encoding block adjustmentmethod in that the basic encoding block need not be adjusted byexpanding the size of the image and only the first adjustment can beperformed although the basic encoding block is adjusted.

Also, the basic encoding block adjustment method may performencoding/decoding on the existing image data and may not requiregeneration of information (size and split information of a readjustedblock, etc.) regarding the basic encoding block of the picture. In otherwords, a process of encoding/decoding unnecessary data may not berequired. However, in the basic encoding block adjustment method, avariable basic encoding block is configured on the boundary of thepicture. Thus, exceptional processing is required. As a result, theuniformity of the encoding/decoding structure deteriorates.

On the other hand, the image size adjustment method may maintain theuniformity of the encoding/decoding structure when the picture isexpanded to an integral multiple of the basic encoding block. However,unnecessary data generated during the expanding process should beencoded/decoded, which may cause a reduction in encoding performance.

FIG. 15 is a flowchart showing a method of encoding an image byadjusting the size of the image according to an embodiment of thepresent invention.

Referring to FIG. 15 , the image encoding apparatus for adjusting animage size according to an embodiment of the present invention may setsize information of a picture and size information and block splitinformation of a basic encoding block (S1510) and may expand the size ofthe picture in accordance with a reset basic encoding block on the basisof the block split information or the integral multiple of the basicencoding block (S1520). Also, the image encoding apparatus may partitionthe expanded picture into basic encoding blocks or reset basic encodingblocks depending on a block location (S1530), may encode the basicencoding blocks in a block scan order (S1540), and may transmit abitstream corresponding to encoding to an image decoding apparatus.

Also, the image decoding apparatus according to an embodiment of thepresent invention may operate corresponding to the above-described imageencoding apparatus. In other words, the image decoding apparatus mayrestore the size information of the picture and the size information andthe block split information of the basic encoding block on the basis ofthe bitstream received from the image encoding apparatus and may expandthe size of the picture in accordance with the reset basic encodingblock on the basis of the block split information or the integralmultiple of the size of the basic encoding block. Also, the imagedecoding apparatus may partition the expanded picture into the basicencoding blocks or the reset basic encoding blocks and may decode thebasic encoding blocks in the block scan order.

Here, the expansion of the picture may be performed in some directions,such as to the right of or downward from the picture, in accordance withthe integral multiple of the basic encoding block or in accordance witha block size supported according to block split settings supported bythe block split unit.

Also, the scan order has been described as being reset according to thenumber of basic encoding blocks. However, the scan order may not bechanged because the number of blocks is not changed, that is, one block,even though the basic encoding block is reset.

3. Adaptive Image Data Processing Method

The adaptive image data processing method is a method of expanding thesize of the image and adaptively processing image encoding/decoding dataof the expanded region on the basis of the region of the existing imagewhen the size of the picture is not an integral multiple of the size ofthe basic encoding block. In other words, the adaptive image dataprocessing method is a method of explicitly processing the imageencoding/decoding data based on the region of the existing image andimplicitly processing image encoding/decoding data based on the expandedregion.

FIGS. 16A to 16D show an example diagram for illustrating an adaptiveimage data processing method according to an embodiment of the presentinvention.

Referring to FIG. 16A, it is assumed that the picture has a size of392×272 and the basic encoding block has a size of 64×64. Thus, thepicture may have a remaining horizontal length of 8 on the rightboundary and a remaining vertical length of 16 on the lower boundary andmay be horizontally expanded by 56 and vertically expanded by 48. Theexpanded picture may have a size of 448×320 and may be an integralmultiple of the size of the basic encoding block.

A method of explicitly and implicitly processing split informationgenerated to acquire an optical split form from the basic encoding blockaccording to the split settings will be described below.

For convenience of description, each process to be described below maybe represented with an ordering number in parentheses of the block, anda number following “-” along with the ordering number in the parenthesesmay refer to the location of a block in which a corresponding process isperformed. Here, the number denoting the location of the block mayinclude indices 1, 2, 3, and 4 allocated in a z scan order (an order ofupper left, upper right, lower left, and lower right) when four blocksare acquired according to a quad tree-based split and may includeindices 1 and 2 allocated in a vertical or horizontal order when twoblocks are acquired according to a binary tree-based split. For example,when four blocks are acquired according to the quad tree-based split,(2-1) may refer to a second process and may denote that the process isperformed by an upper left block among the four blocks.

Referring to FIG. 16B, an actual image may be a hatched region (with asize of 8×16) on the upper left of the block, and the remaining regionother than the hatched region in the 64×64 block may be an expandedregion and may refer to a padded region when the hatched region isfilled through outer pixel padding. Here, it is assumed that a splitbased on a single tree such as a quad tree may be supported for thebasic encoding block, and 64×64, 32×32, 16×16, and 8×8 blocks may beacquired through the block split unit when the minimum block has a sizeof 8×8.

The actual image may be composed of two 8×8 blocks among the blockscapable of being supported by the block split unit, and block splitinformation may be adaptively configured according to the followingprocess.

A first process is to acquire candidates by splitting a 64×64 block (1).Here, 64×64 and 32×32 blocks may be acquirable, but an unconditionalsplit should be performed in order to acquire a block including onlyactual image data. Thus, the 64×64 block, which includes the paddeddata, may be excluded from the candidates. Accordingly, only 32×32blocks may be candidates. Here, the split information may indicate 1 or0 depending on the presence of the split, but one 32×32 block is acandidate. Thus, the unconditional split should be performed.Accordingly, the split information may be explicitly or implicitlyprocessed depending on the value.

A second process is to acquire candidates by splitting an upper leftblock (2-1), an upper right block (2-2), a lower left block (2-3), and alower right block (2-4), which have a size of 32×32. Here, 32×32 and16×16 blocks are acquirable from each block. However, for the upper leftblock (2-1), only the 16×16 block may be a candidate for the same reasonas in the first process. Also, for the upper right block (2-2), thelower left block (2-3), and the lower right block (2-4), only the 32×32block may be a candidate, because only a padded region is included andthus an additional split is meaningless. Accordingly, since only onecandidate is present for each block, the split information may beimplicitly processed.

A third process is to acquire candidates by splitting an upper leftblock (3-1), an upper right block (3-2), a lower left block (3-3), and alower right block (3-4), which have a size of 16×16. Here, 16×16 and 8×8blocks are acquirable from each block. However, for the upper left block(3-1), only the 8×8 block may be a candidate for the same reason as inthe first process and the second process. For the upper right block(3-2), the lower left block (3-3), and the lower right block (3-4), onlythe 16×16 block may be a candidate for the same reason as the upperright block (2-2), the lower left block (2-3), and the lower right block(2-4) of the second process. Accordingly, since only one candidate ispresent for each block, the split information may be implicitlyprocessed.

A fourth process is to acquire four 8×8 blocks by splitting the upperleft block (301) of the third process. A block including actual imagedata, which has a size of 8×16, is greater than blocks obtained bysplitting the upper left block (3-1), which have a size of 8×8, but maybe composed of upper left and lower left 8×8 blocks. Thus, an additionalsplit may not be performed.

The split information generated during the above first to fourthprocesses may be expressed as (8×8): 1-1-1-0-0-0-0-0-0 and may indicateinformation regarding the first process, the upper left block of thesecond process, the upper left block of the third process, the upperright block of the third process, the lower left block of the thirdprocess, the lower right block of the third process, the upper rightblock of the second process, the lower left block of the second process,and the lower right block of the second process in sequence. Here, 1 and0 of the split information may be a syntax element indicating thepresence of the split according to the quad tree-based split.

However, the split information generated during the above first tofourth processes may be implicitly processed. Accordingly, when adaptiveencoding/decoding data is processed on the basis of the actual imagedata, there is no process of explicitly processing the data, and thusthe split information may not be present. This may correspond to a casein which an optimal split form may be checked without generation andrestoration of the split information.

Referring to FIG. 16C, an actual image may be a hatched region (with asize of 64×16) on the upper of the block, and the remaining region otherthan the hatched region in the 64×64 block may be an expanded region andmay refer to a padded region when the hatched region is filled throughouter pixel padding. Here, it is assumed that a split based on a singletree such as a binary tree may be supported for the basic encodingblock, and 64×64, 64×32, 32×64, 64×16, 16×64, 32×32, 64×8, 8×64, 32×16,and 16×32 blocks may be acquired through the block split unit when theminimum block has a side length of 8 and the maximum split depth is 3.Here, the block split information may be adaptively configured accordingto the following process.

A first process is to acquire candidates by splitting a 64×64 block (1).Here, a split depth may increase from 0 to 1, and 64×64, 64×32, and32×64 blocks may be acquirable, but an unconditional split should beperformed in order to acquire a block including only actual image data.Thus, the 64×64 block, which includes the padded data, may be excludedfrom the candidates. Also, the 32×64 block may be excluded from thecandidates because an additional split, such as a horizontal split ofeach block, is required in subsequent steps in order to acquire a blockincluding only the actual image data. In other words, since the numberof splits increases, the 32×64 block may be excluded from thecandidates. Accordingly, only the 64×32 block may be included in thecandidates, and only one candidate is present for each block. Thus, thesplit information may be implicitly processed.

A second process is to acquire candidates by splitting an upper block(2-1) and a lower block (2-2), which have a size of 64×32. Here, thesplit depth may increase from 1 to 2, and 64×32, 64×16, and 32×32 blocksare acquirable from each block. However, for the upper block (2-1), onlythe 64×16 block may be a candidate because the 64×32 and 32×32 blocksmay be excluded from the candidates for the same reason as in the firstprocess. Also, for the lower block (2-2), only the 64×32 block may be acandidate because only a padded region is included and thus anadditional split is meaningless. Accordingly, since only one candidateis present for each block, the split information may be implicitlyprocessed.

A third process is to acquire candidates by splitting an upper block(3-1) and a lower block (3-2), which have a size of 64×16. Here, thesplit depth may increase from 2 to 3, and 64×16, 64×8, and 32×16 blocksare acquirable from each block. However, for the upper block (3-1), oneof the 64×16, 64×8, and 32×16 blocks may be determined as an optimalsplit form because a block including only actual image data is acquired.Also, for the lower block (3-2), only the 64×16 block may be a candidatebecause only a padded region is included and thus an additional split ismeaningless.

A fourth process is to acquire candidates by splitting the upper block(3-1) of the third process. However, the optimal split form of the upperblock (3-1) of the third process is determined, and the side length, thesplit depth, and the like reach the minimum block conditions. Thus, anadditional split may not be performed.

The split information generated during the above first to fourthprocesses may be expressed as (64×16): 1-0-1-0-0-0-0 when the 64×16block is determined as the optimal split form, (64×8): 1-0-1-0-1-0-0-0when the 64×8 block is determined as the optimal split form, and(32×16): 1-0-1-0-1-1-0-0 when the 32×16 block is determined as theoptimal split form. Here, the split information may be informationregarding the first process, the upper block of the second process, theupper block of the third process, the lower block of the third process,and the lower block of the second process in sequence.

In detail, the split information may include two syntax elementsindicating split presence and a split direction in the first process,two syntax elements indicating split presence and a split direction inthe upper block of the second process, two syntax elements indicatingsplit presence and a split direction in the upper block of the thirdprocess, one syntax element indicating split presence in the lower blockof the third process, and one syntax element indicating split presencein the lower block of the second process. When the syntax elementregarding the presence of the split is 0, a split is not performed. Whenthe syntax element regarding the presence of the split is 1, a split isperformed. When the syntax element regarding the split direction is 0,the split direction may indicate a horizontal direction. When the syntaxelement regarding the split direction is 1, the split direction mayindicate a vertical direction.

However, among the split information generated during the above first tofourth processes, the upper block of the third process may be explicitlyprocessed, and the other blocks may be implicitly processed.Accordingly, when adaptive encoding/decoding data is processed on thebasis of the actual image data, the split information may be expressedas (64×16): 0 when the 64×16 block is determined as the optimal splitform, (64×8): 1-0 when the 64×8 block is determined as the optimal splitform, and (32×16): 1-1 when the 32×16 block is determined as the optimalsplit form.

In other words, by using the split information that is generated in theupper block of the third process and explicitly processed, other thanthe split information that is implicitly processed, the optimal splitform may be checked in the basic encoding block. However, a case inwhich some of the split information is implicitly processed has beendescribed above, but when 64×16 satisfies the minimum block conditionsaccording to the block split settings, all the split information may beimplicitly processed.

Referring to FIG. 16D, an actual image may be a hatched region (with asize of 8×64) to the left of the block, and the remaining region otherthan the hatched region in the 64864 block may be an expanded region andmay refer to a padded region when the hatched region is filled throughouter pixel padding. Here, splits based on multiple trees such as a quadtree and a binary tree may be supported for the basic encoding block.For split settings of the quad tree, the maximum block may have the samesize, which is 64×64, as the size of the basic encoding block, and theminimum block may have a size of 16×16. For split settings of the binarytree, the maximum block may have a size of 32×32, the minimum block mayhave a side length of 4, and the maximum split depth may be 3. Inaddition, it is assumed that the tree splits may be performed accordingto priorities (e.g., the quad tree-based split takes precedence over thebinary tree-based split) when ranges generated by the tree splitsoverlap each other and that 64×64, 32×32, 32×16, 16×32, 32×8, 8×32,16×16, 32×4, 4×32, 16×8, and 8×16 blocks may be acquired through theblock split unit. Here, the block split information may be adaptivelyconfigured according to the following process.

A first process is to acquire candidates by splitting a 64×64 block (1).Here, 64×64 and 32×32 blocks may be acquirable, but an unconditionalsplit should be performed in order to acquire a block including onlyactual image data. Thus, the 64×64 block, which includes the paddeddata, may be excluded from the candidates. In the existing case, splitinformation such as information regarding the presence of the quadtree-based split, the presence of the binary tree-based split, and thedirection of the binary tree-based split may be generated. However,according to an embodiment, only one candidate according to the quadtree-based split may be supported, and thus the split information may beimplicitly processed.

A second process is to acquire candidates by splitting an upper leftblock (2-1), an upper right block (2-2), a lower left block (2-3), and alower right block (2-4), which have a size of 32×32. Here, 32×32 and16×16 blocks may be acquirable from each block when the quad tree-basedsplit is performed, 32×16 and 16×32 blocks may be acquirable from eachblock, and the split depth increases from 0 to 1 when the binarytree-based split is performed.

For the upper lift block (2-1) and the lower left block (2-3), the 32×32block among the candidate blocks should be unconditionally split andthus may be excluded from a group of the candidates, and also the 32×16and 16×16 blocks may require an additional split such as a verticalsplit of each block in subsequent steps in order to acquire a blockincluding actual image data and may be excluded from the group of thecandidates. Here, the size of the block including the actual image datamay be 8×64. However, the quad tree-based split is performed in thefirst process, and thus a size of 8×32 may be regarded as the size ofthe block including the actual image data. Accordingly, only one 16×32block may be present as a candidate for each block. However, the 16×32block requires an additional split in subsequent steps, but a smallernumber of splits are performed to reach the block including the actualimage data compared to the other candidate blocks. Thus, the 16×32 blockmay be an optimal candidate.

According to split information for acquiring the optimal candidate(i.e., the 16×32 block), a quad tree-based split having a higherpriority among the splits based on multiple trees is not performed, anda binary tree-based split having the next higher priority is performed.The split may have a vertical direction. Here, the split informationincludes split direction information and split presence information inthe binary tree-based split. However, when the quad tree-based split andthe binary tree-based split overlap each other, the quad tree-basedsplit having a higher priority has a 32×32 block as a candidate, andthus the binary tree may not require the 32×32 block to be supported asa candidate. In other words, when the quad tree-based split and thebinary tree-based split overlap each other, the split presenceinformation for the binary tree-based split may be excluded from thesplit information, and only the split direction information may beincluded in the split information. Accordingly, since only one optimalcandidate is present, the split information may be implicitly processed.

The upper right block (2-2) and the lower right block (2-4) include onlya padded region. Thus, an additional split is meaningless, and only the32×32 block may be a candidate. Accordingly, since only one candidate ispresent, the split information may be implicitly processed.

The third process is to acquire candidates by splitting a left block(3-1-1) and a right block (3-1-2), which has a size of 16×32 and whichis acquired from the upper left block (2-1) and by splitting a leftblock (3-2-1) and a right block (3-2-2) which has a size of 16×32 andwhich is acquired from the lower left block (2-3). Here, the split depthmay increase from 1 to 2, and the 16×32, 16×16, and 8×32 blocks may beacquirable from each block.

For the left block (3-1-1), the 16×32 and 16×16 blocks may be excludedfrom the candidate group for the same reason as that of the upper leftblock (2-1) of the second process, and only the 8×32 block may be acandidate. Here, conventionally, the split information includes splitpresence information and split direction information in the binarytree-based split, but two vertical splits are unconditionally performed.Thus, the split information may be implicitly processed. The 8×32 blockacquired through the above-described process may be a block includingonly the actual image data.

The right block (3-1-2) includes only a padded region. Thus, anadditional split is meaningless, and only the 16×32 block may be acandidate. Accordingly, since only one candidate is present, the splitinformation may be implicitly processed.

For the left block (3-2-1), the split information may be implicitlyprocessed through the same process as that of the left block (3-1-1),and the acquired 8×32 block may be a block including only the actualimage data.

The right block (3-2-2) includes only a padded region. Thus, anadditional split is meaningless, and only the 16×32 block may be acandidate. Accordingly, since only one candidate is present, the splitinformation may be implicitly processed.

A fourth process is to acquire candidates by splitting a left block(4-1-1) and a right block (4-1-2), which have a size of 8×32 and whichis acquired from the left block (3-1-1) and splitting a left block(4-2-1) and a right block (4-2-2) which have a size of 8×32 and isacquired from the left block (3-2-1). Here, the split depth may increasefrom 2 to 3, and 8×32, 8×16, and 4×32 blocks may be acquirable from eachblock. However, for the left block (4-1-1) and the left block (4-2-1),one of the 8×32, 8×8, and 4×32 blocks may be determined as an optimalsplit form because a block including only actual image data is acquired.Also, for the right block (4-1-2) and the right block (4-2-2), only the8×32 block may be a candidate because only a padded region is includedand thus an additional split is meaningless. Here, for the left block(4-1-1) and the left block (4-2-1), the split information may beexplicitly processed according to the determination of the optimal splitform. For the right block (4-1-2) and the right block (4-2-2), the splitinformation may be implicitly processed because only one candidate ispresent.

The split information generated during the above first to fourthprocesses may be expressed as (8×32): 1-0-1-1-1-0-0-0-0-0-1-1-1-0-0-0-0when the 8×32 block is determined as the optimal split form, (8×16):1-0-1-1-1-1-0-0-0-0-0-1-1-1-0-0-0-0 when the 8×16 block is determined asthe optimal split form, and (4×32): 1-0-1-1-1-1-1-0-0-0-0-1-1-1-0-0-0-0when the 4×32 block is determined as the optimal split form. Here, thesplit information may be information regarding the first process, theupper left block (2-1) of the second process, the left block (3-1-1) ofthe third process, the left block (4-1-1) of the fourth process, theright block (4-1-2) of the fourth process, the right block (3-1-2) ofthe third process, the upper right block (2-2) of the second process,the lower left block (2-3) of the second process, the left block (3-2-1)of the third process, the left block (4-2-1) of the fourth process, theright block (4-2-2) of the fourth process, the right block (3-2-2) ofthe third process, and the lower right block (2-4) of the second processin sequence.

In detail, the split information may include one syntax elementindicating split presence in the first process, two syntax elementsindicating split presence and a split direction in the upper left block(2-1) of the second process, two syntax elements indicating splitpresence and a split direction in the left block (3-1-1) of the thirdprocess, one syntax element indicating split presence in the left block(4-1-1) of the fourth process, one syntax element indicating splitpresence in the left block (4-1-2) of the fourth process, one syntaxelement indicating split presence in the right block (3-1-2) of thethird process, one syntax element indicating split presence in the upperright block (2-2) of the second process, two syntax elements indicatingsplit presence and a split direction in the left block (3-2-1) of thethird process, one syntax element indicating split presence in the leftblock (4-2-1) of the fourth process, one syntax element indicating splitpresence in the left block (4-2-2) of the fourth process, one syntaxelement indicating split presence in the right block (3-2-2) of thethird process, and one syntax element indicating split presence in thelower right block (2-4) of the second process. When the syntax elementregarding the presence of the split is 0, a split is not performed. Whenthe syntax element regarding the presence of the split is 1, a split isperformed. When the syntax element regarding the split direction is 0,the split direction may indicate a horizontal direction. When the syntaxelement regarding the split direction is 1, the split direction mayindicate a vertical direction.

However, among the split information generated during the above first tofourth processes, the left blocks (4-1-1 and 4-2-1) of the fourthprocess may be explicitly processed, and the other blocks may beimplicitly processed. Accordingly, when adaptive encoding/decoding datais processed on the basis of the actual image data, the splitinformation may be expressed as (8×32): 0-0 when the 8×32 block isdetermined as the optimal split form, (8×16): 1-0-0 when the 8×16 blockis determined as the optimal split form, and (4×32): 1-1-1 when the 4×32block is determined as the optimal split form.

In other words, by using the split information that is generated in theleft blocks (4-1-1 and 4-2-1) of the fourth process and explicitlyprocessed, other than the split information that is implicitlyprocessed, the optimal split form may be checked in the basic encodingblock.

A case in which the adaptive image data processing method is differentlyapplied will be described below.

4-1. Adaptive Image Data Processing Method 1 According to a Plurality ofSplit Methods

FIGS. 17A to 17F show a first example diagram illustrating an adaptiveimage data processing method using a plurality of split methodsaccording to an embodiment of the present invention.

The quad tree-based split and the binary tree-based split will bedescribed again. The quad tree-based split may refer to a tree-basedsplit with non-directionality and a split in which a plurality ofacquired sub-blocks do not have specific directionality, and the binarytree-based split may refer to a tree-based split with directionality anda split in which a plurality of acquired sub-blocks have specificdirectionality such as a horizontal or vertical direction. Accordingly,for the binary tree, information regarding the split direction may beadditionally generated for the split information.

Splits based on multiple trees may refer to a scheme in which both ofthe quad tree-based split and the binary tree-based split are supported.However, splits based on other trees may be supported, and thus thepresent invention is not limited thereto.

Here, the splits based on multiple trees may be supported for a 64×64block. In split settings for the quad tree, the maximum block may havethe same size, which is 64×64, as the basic encoding block and theminimum block may have a size of 16×16. In split settings for the binarytree, the maximum block may have a size of 32×32, the minimum block mayhave a side length of 4, and the maximum split depth may be 3. Also, apriority may be present for each tree-based split. The quad tree-basedsplit may have a higher priority when the quad tree-based split and thebinary tree-based split overlap with each other, but the presentinvention is not limited thereto.

Referring to FIG. 17A, actual image data in a block may be a hatchedregion (with a size of 32×16). First, the quad tree-based split may beperformed on a block (1), and thus four 32×32 blocks may be acquired.Also, the binary tree-based split may be performed on an upper leftblock (2), which is one of the four 32×32 blocks, and thus two 32×16blocks may be acquired. Here, an upper block (3), which is one of thetwo 32×16 blocks, may include only actual image data, and the splitinformation may be implicitly processed as described above. In otherwords, the split information may be adaptively processed.

Referring to FIG. 17B, actual image data in a block may be a hatchedregion (with a size of 8×8). First, the quad tree-based split may beperformed on a block (1), and thus four 32×32 blocks may be acquired.Also, the quad tree-based split may be performed on an upper left block(2), which is one of the four 32×32 blocks, and thus four 16×16 blocksmay be acquired. The binary tree-based split may be vertically performedon an upper left block (3), which is one of the four 16×16 blocks, andthus two 8×16 blocks may be acquired. Also, the binary tree-based splitmay be horizontally performed on a left block (4), which is one of thetwo 8×16 blocks, and thus two 8×8 blocks may be acquired. Here, an upperblock (5), which is one of the two 8×8 blocks, may include only actualimage data, and the split information may be implicitly processed asdescribed above. In other words, the split information may be adaptivelyprocessed.

Referring to FIG. 17C, actual image data in a block may be a hatchedregion (with a size of 8×8), as shown in FIG. 17B. First, the quadtree-based split may be performed on a block (1), and thus four 32×32blocks may be acquired. Also, the quad tree-based split may be performedon an upper left block (2), which is one of the four 32×32 blocks, andthus four 16×16 blocks may be acquired. The quad tree-based split may beperformed on an upper left block (3), which is one of the four 16×16blocks, and thus four 8×8 blocks may be acquired. Here, an upper leftblock (4), which is one of the four 8×8 blocks, may include only actualimage data, and the split information may be implicitly processed asdescribed above. In other words, the split information may be adaptivelyprocessed.

Referring to FIGS. 17B and 17C, the two blocks may have the same actualimage data region. However, in order to acquire the block (5), twobinary tree splits may be performed on the block (3) in FIG. 17B, andone quad tree split may be performed on the block (3) in FIG. 17C.Accordingly, the case of FIG. 17C may be more efficient considering thenumber of splits.

However, since the minimum block of the quad tree-based split has a sizeof 16×16, the last quad tree-based split in FIG. 14 c may be restricted.In other words, the quad tree-based split cannot be performed on acorresponding block, and only the binary tree-based split may beperformed.

In order to solve such a problem, block split settings for a basicencoding block belonging to a boundary of the picture may beexceptionally set to be different from block split settings for a basicencoding block inside the picture. For example, the quad tree-basedsplit may be allowed for a basic encoding block located on a lower rightboundary of the picture when horizontal and vertical lengths of theregion including the actual image data do not exceed ½ of horizontal andvertical lengths of the block before the split.

According to the above-described exceptional block split settings, inthe block shown in FIG. 17C, the region including the actual image datahas a size of 8×8 and the block before the split has a size of 16×16. Inthis case, the size of the region including the actual image data doesnot exceed ½ of the size of the block before the split, and thus thequad tree-based split may be third performed. In other words, FIG. 17Cmay show a case in which the size of the minimum block of the quad treeis changed to 888 in some block split settings.

Referring to FIG. 17D, actual image data in a block may be a hatchedregion (with a size of 48×8). First, the quad tree-based split may beperformed on a block (1), and thus four 32×32 blocks may be acquired.Here, since an upper left block (2-1) and an upper right block (2-2),each of which is one of the four 32×32 blocks, include actual imagedata, the upper left bock (2-1) and the upper right block (2-2) may beadditionally and individually split.

Specifically, with respect to the additional splits of the two blocks,the binary tree-based split may be horizontally performed on the upperleft block (2-1), and thus two 32×16 blocks may be acquired. The binarytree-based split may be horizontally performed on an upper block (3-1),which is one of the two 32×16 blocks, and thus four 32×8 blocks may beacquired. Here, an upper block (4-1), which is one of the 32×8 blocks,may include only actual image data.

The binary tree-based split may be horizontally performed on the upperright block (2-2), and thus two 32×16 blocks may be acquired. The binarytree-based split may be vertically performed on an upper block (3-2),which is one of the two 32×16 blocks, and thus two 16×16 blocks may beacquired. Also, the binary tree-based split may be horizontallyperformed on a left block (4), which is one of the two 16×16 blocks, andthus two 16×8 blocks may be acquired. Here, an upper block (5), which isone of the 16×8 blocks, may include only actual image data. In otherwords, the number of splits of the block shown in FIG. 17D may increasein order to acquire a block including only actual image data, which maycause an increase in complexity.

Referring to FIG. 17E, actual image data in a block may be a hatchedregion (with a size of 48×8), as shown in FIG. 17D. The binarytree-based split may be horizontally performed on a block (1), and thustwo 64×32 blocks may be acquired. The binary tree-based split may behorizontally performed on an upper block (2-2), which is one of the two64×32 blocks, and thus two 64×16 blocks may be acquired. The binarytree-based split may be horizontally performed on an upper block (3),which is one of the two 64×16 blocks, and thus two 64×8 blocks may beacquired. Even when a block including only actual image data isacquired, this process may be efficient because a smaller number ofsplits are required.

However, due to block split settings, only the quad tree-based split maybe supported in an initial block split stage. Thus, in an initial block,the binary tree-based split cannot be performed, and only the quadtree-based split may be allowed.

In order to solve such a problem, block split settings for a basicencoding block belonging to a boundary of the picture may beexceptionally set to be different from block split settings for a basicencoding block inside the picture. For example, the binary tree-basedsplit may be allowed for a basic encoding block located on a lower rightboundary of the picture when horizontal and vertical lengths of theregion including the actual image data exceed ½ of horizontal andvertical lengths of the block before the split. Here, the horizontalbinary tree-based split may be allowed when the horizontal lengthexceeds ½, and the vertical binary tree-based split may be allowed whenthe vertical length exceeds ½. However, the above-described splitsettings may be applied in common to a tree-based split requiring adirection, such as a ternary tree and an asymmetric binary tree, andthus the present invention is not limited thereto.

According to the above-described exceptional block split settings, inthe block shown in FIG. 17E, the region including the actual image datahas a size of 48×8 and the block before the split has a size of 64×64.In this case, the size of the region including the actual image dataexceeds ½ of the size of the block before the split, and thus the binarytree-based split may be performed. In other words, FIG. 17E may show acase in which the size of the maximum block of the binary tree ischanged to 64×64 in some block split settings.

Here, when the horizontal and vertical lengths of the region includingthe actual image data exceed ½ of the horizontal and vertical lengths ofthe block before the split, the quad tree-based split may require asmaller number of splits than the binary tree-based split.

The above-described block split settings may allow the quad tree-basedsplit when both of the horizontal and vertical lengths of the regionincluding the actual image data on the upper right boundary of thepicture exceed ½ of the horizontal and vertical lengths of the blockbefore the split and when both of the horizontal and vertical lengths ofthe region are less than or equal to ½ of the horizontal and verticallengths of the block before the split and also may allow the horizontalor vertical binary tree-based split when only one of the horizontal andvertical lengths of the region exceeds ½ of a corresponding one of thehorizontal and vertical lengths of the block before the split. When theabove-described block split settings do not allow the split, exceptionalprocessing of the corresponding block settings (a change in some of thesplit settings) may be possible. Accordingly, a flexible block split maybe possible depending on the changed split settings.

The block split settings may allow the following splits on boundariesother than some boundaries. The block split settings may allow thehorizontal binary tree-based split regardless of the vertical length ofthe region including the actual image data on the lower boundary of thepicture and may allow the vertical binary tree-based split regardless ofthe horizontal length of the region including the actual image data onthe right boundary of the picture.

In other words, the quad tree-based or binary tree-based split may bepossible on the lower right boundary of the picture. Also, on the rightboundary and the lower boundary of the picture, the binary tree-basedsplit may be possible in a split direction parallel to the boundaries,and the quad tree-based split may also be possible. However, the binarytree-based split may be impossible in a split direction perpendicular tothe boundaries. The description of the present invention assumes thatthe block before the split includes a padded image in addition to actualimage data. Accordingly, the description cannot be applied when theblock before the split includes only actual image data.

When the maximum block size of the binary tree-based split is 32×32,blocks acquirable in accordance with minimum block size settings (a sidelength of 4 and a maximum split depth of 3) may have sizes of 32×32,32×16, 16×32, 32×8, 8×32, 16×16, 32×4, 4×32, 16×8, and 8×16. However, asshown in FIG. 17E, when the maximum block size of the binary tree-basedsplit is changed from 32×32 to 64×64, the acquirable blocks may havesizes of 64×64, 64×32, 32×64, 64×16, 16×64, 32×32, 64×8, 8×64, 32×16,and 16×32. Accordingly, some of the acquirable blocks in the existingblock split settings cannot be acquired from the changed block splitsettings.

Here, different block split settings such as the side length of theminimum block or the maximum split depth, in addition to the size of themaximum block, may be changed. Thus, a block split may be performed on abasic encoding block on a boundary of the picture.

Referring to FIG. 17F, actual image data in a block may be a hatchedregion (with a size of 48×8), as shown in FIGS. 17D and 17E. Accordingto the above-described block split settings, the binary tree-based splitmay be horizontally performed on a block (1), and thus two 64×32 blocksmay be acquired. The binary tree-based split may be horizontallyperformed on an upper block (2-2), which is one of the two 64×32 blocks,and thus two 64×16 blocks may be acquired. The binary tree-based splitmay be horizontally performed on an upper block (3), which is one of thetwo 64×16 blocks, and thus two 64×8 blocks may be acquired. Also, thebinary tree-based split may be vertically performed on an upper block(4), which is one of the two 64×8 blocks, and thus two 32×8 blocks maybe acquired. A left block, which is one of the two 32×8 blocks, mayinclude only actual image data, and the binary tree-based split may beperformed on a right block (5), which is the other of the two 32×8blocks, and thus two 16×8 blocks may be acquired. A left block (6),which is one of the two 16×8 blocks, may include only actual image data.

However, even when an exceptional case, such as the above-describedblock split settings, is considered, a split of less than the minimumsize for encoding residual coefficients cannot be allowed. For example,when the minimum size for encoding residual coefficients is 4×4, a blockwhich is smaller than a 4×4 block cannot be allowed. However, the blockmay be allowed when horizontal and vertical lengths are greater than orequal to the horizontal and vertical lengths of the minimum size forencoding residual coefficients or when one of the horizontal andvertical lengths is shorter but a product between the horizontal lengthand the vertical length is greater than or equal to a product betweenthe horizontal length and the vertical length of the minimum size forencoding residual coefficients because the other of the horizontal andvertical lengths is sufficiently long. Here, a flag for checking whethera residual coefficient is present (e.g., a coded block flag) or the likemay be supported for the block.

4-2. Adaptive Image Data Processing Method 2 According to a Plurality ofSplit Methods

FIGS. 18A to 18C show a second example diagram illustrating an adaptiveimage data processing method using a plurality of split methodsaccording to an embodiment of the present invention.

Here, it is assumed that splits based on multiple trees such as a binarytree and a ternary tree may be supported for a block, the maximum blockhas a size of 64×64, the minimum block has a side length of 4, and themaximum split depth is 4. A priority is not present for each tree-basedsplit, and selection information regarding which tree-based split is tobe selected may be generated.

Accordingly, split presence information, tree selection information, andsplit direction information may be sequentially generated as the splitinformation. Here, the split is not performed when the split presenceinformation is 0 and is performed when the split presence informationis 1. When the tree selection information is 0, the tree selectioninformation may indicate the binary tree-based split. When the treeselection information is 1, the tree selection information may indicatethe ternary tree-based split. Also, when the split direction informationis 0, the split direction information may indicate a horizontaldirection. When the split direction information is 1, the splitdirection information may indicate a vertical direction.

Also, it is assumed that the exceptional block split settings describedin the above-described adaptive image data processing method 1 accordingto the plurality of split methods may be applied to the adaptive imagedata processing method 2.

Referring to FIGS. 18A to 18C, a basic encoding block may have a size of64×64, and actual image data may be a hatched region (with a size of64×16). Here, since the binary tree-based split and the ternarytree-based split may be performed on the block, blocks with sizes of64×64, 64×32, 32×64, 64×16/64×32/64×16, and 16×64/32×64/16×64 areacquirable through one split.

Referring to FIG. 18A, the binary tree-based split may be horizontallyperformed on a block (1), and thus two 64×32 blocks may be acquired.Here, the split depth may increase from 0 to 1, and the splitinformation may be generated as 1-0-0 or may be implicitly processed.The binary tree-based split may be horizontally performed on an upperblock (2), which is one of the two 64×32 blocks, and thus two 64×16blocks may be acquired. Here, the split depth may increase from 1 to 2,and the split information may be generated as 1-0-0 or may be implicitlyprocessed. An upper block (3), which is one of the two 64×16 blocks,includes only actual image data, and thus an optimal split form may bedetermined through an additional split. However, it is assumed that thisregion is not additionally split as shown in FIGS. 18B and 18 c as wellas this example.

The split information generated during the above split process may beexpressed as (64×16): 1-0-0-1-0-0-0-0-0. However, split informationcorresponding to the upper block (3), which is one of the two 64×16blocks and includes actual image data, may be explicitly processed, andsplit information corresponding to the other blocks, which include onlya padded region, may be implicitly processed. Accordingly, when adaptiveencoding/decoding data is processed on the basis of the actual imagedata, the split information may be 0 depending on an explicitlyprocessing process. Here, 0 may denote split presence information ofsplit information that is third generated.

Referring to FIG. 18B, the ternary tree-based split may be horizontallyperformed on a block (1), and thus two 64×16 blocks and one 64×32 blockmay be acquired. Here, the split depth may increase from 0 to 1, and thesplit information may be generated as 1-1-0 or may be implicitlyprocessed. Since an upper block (2), which is one of the two 64×16blocks, may include only actual image data, an additional split may notbe required. Accordingly, an optimal split form may be determined.However, the optimal split form may be determined through the additionalsplit.

The split information generated during the above split process may beexpressed as (64×16): 1-1-0-0-0-0. However, split informationcorresponding to the upper block (2), which is one of the two 64×16blocks and includes actual image data, may be explicitly processed, andsplit information corresponding to the other blocks, which include onlya padded region, may be implicitly processed. Accordingly, when adaptiveencoding/decoding data is processed on the basis of the actual imagedata, the split information may be 0 depending on an explicitlyprocessing process. Here, 0 may denote split presence information ofsplit information that is second generated.

In FIGS. 18A and 18B, the split information may be the same, that is, 0.However, there may be a difference in the step of reaching the blockincluding only the actual image data even though the data is implicitlyprocessed as the number of splits increases in the encoding/decodingprocessing process. Accordingly, when there are one or more methods ofreaching the block including only the actual image data from an initialblock or a basic encoding block, a method using a smaller number ofsplits may be set to be implicitly determined. In other words, when bothof the cases of FIGS. 18A and 18B are possible, a method using theternary tree-based split shown in FIG. 18B may be set to be implicitlydetermined because the case of FIG. 18B requires a smaller number ofsplits.

According to an embodiment, when the horizontal length of the regionincluding the actual image data is equal to ¼ or ¾ of the horizontallength of the block before the split on the right boundary of thepicture, the ternary tree-based split may be vertically performed. Whenthe horizontal length of the region is not equal to ¼ or ¾ of thehorizontal length of the horizontal length of the block before thesplit, the binary tree-based split may be vertically performed.

Also, when the vertical length of the region including the actual imagedata is equal to ¼ or ¾ of the vertical length of the block before thesplit on a lower boundary of the picture, the ternary tree-based splitmay be horizontally performed. When the vertical length of the region isnot equal to ¼ or ¾ of the horizontal length of the vertical length ofthe block before the split, the binary tree-based split may behorizontally performed.

The above description is associated with some boundaries, and thus asplit on the other boundaries may be as follows.

Also, when at least one of the horizontal and vertical lengths of theregion including the actual image data is equal to ¼ or ¾ of acorresponding one of the horizontal and vertical lengths of the blockbefore the split on a lower right boundary of the picture, the ternarytree-based split may be horizontally or vertically performed. When thehorizontal and vertical lengths of the region are not equal to ¼ or ¾ ofthe horizontal and vertical lengths of the block before the split, thebinary tree-based split may be horizontally or vertically performed.

Here, when any one of the horizontal and vertical lengths of the regionincluding the actual image data does not exceed ½, the binary tree-basedsplit may be performed in a direction perpendicular to the correspondinglength. When none of the horizontal and vertical lengths of the regionexceed ½, the binary tree-based split may be horizontally or verticallyperformed. For example, when only the horizontal length of the regionincluding the actual image data does not exceed ½, the binary tree-basedsplit may be vertically performed.

Also, it is determined whether the horizontal length of the regionincluding the actual image data is equal to ¼ or ¾ of the horizontallength of the block before the split or whether the vertical length ofthe region including the actual image data is equal to ¼ or ¾ of thevertical length of the block before the split. When any one issatisfied, the ternary tree-based split may be performed. Otherwise, thebinary tree-based split may be performed.

In summary, the binary tree-based split or the ternary tree-based splitmay be possible on the lower right boundary of the picture. Also, on thelower boundary and the right boundary of the picture, the binarytree-based split or the ternary tree-based split may be possible in asplit direction parallel to the boundaries and is not possible in asplit direction perpendicular to the boundaries.

The above-described embodiments of the present invention may assume thatthe block before the split includes both of actual image data and apadded image, and thus cannot be applied to a case in which only theactual image data is included. For convenience of description, theternary tree-based split has been described as a split into three partsin the above-described ratio (1:2:1), but another ratio may also bepossible depending on encoding settings.

Another embodiment in which a plurality of different split methods areapplied will be described with reference to FIG. 18C, as follows.

Here, it is assumed that splits based on multiple trees such as asymmetric binary tree and an asymmetric binary tree may be supported fora block, the maximum block has a size of 64×64, the minimum block has aside length of 4, and the maximum split depth is 4. Also, a priority isnot present for each tree-based split, and selection informationregarding which tree-based split is to be selected may be generated.Here, for the asymmetric binary tree, split ratio information as well assplit direction information may be additionally required, and apredetermined split ratio, such as 1:3 or 3:1, may be restricted.However, other additional split ratios may be possible.

Accordingly, split presence information, split direction information,and tree selection information may be generated as the splitinformation. When the tree selection information indicates an asymmetricbinary tree, split ratio information may be further generated. Also, thesplit information may be generated in the listed order. Here, the splitis not performed when the split presence information is 0 and isperformed when the split presence information is 1. Also, when the splitdirection information is 0, the split direction information may indicatea horizontal direction. When the split direction information is 1, thesplit direction information may indicate a vertical direction. When thetree selection information is 0, the tree selection information mayindicate the symmetric binary tree-based split. When the tree selectioninformation is 1, the tree selection information may indicate theasymmetric binary tree-based split. Also, when the split ratioinformation according to the asymmetric binary tree-based split is 0,the split ratio information may indicate a ratio having a wide upper orleft side. When the split ratio information is 1, the split ratioinformation may indicate a ratio having a wide lower or right side.

A basic encoding block may have a size of 64×64, and actual image datamay be a hatched region (with a size of 64×16). Here, since thesymmetric binary tree-based split and the asymmetric binary tree-basedsplit may be performed on the block, blocks with sizes of 64×64, 64×32,32×64, 64×48/64×16, 64×16/64×48, 48×64/16×64, and 16×64/48×64 areacquirable.

Referring to FIG. 18C, the asymmetric binary tree-based split may behorizontally performed on a block (1), and thus a 64×16 block and a64×48 block may be acquired. Here, the split depth may increase from 0to 1, and the split information may be generated as 1-0-1-1. However,the split information may be implicitly processed because only onecandidate may be supported. Also, since the 64×16 block (2) may includeonly actual image data, an additional split may not be required.Accordingly, an optimal split form may be determined. However, when anadditional split is performed, the optimal split form may be determinedthrough the additional split according to the block split settings.

The split information generated during the above split process may beexpressed as (64×16): 1-0-1-1-0-0. However, split informationcorresponding to the 64×16 block (2), which includes actual image data,may be explicitly processed, and split information corresponding to theother blocks, which include only a padded region, may be implicitlyprocessed. Accordingly, when adaptive encoding/decoding data isprocessed on the basis of the actual image data, the split informationmay be 0 depending on an explicitly processing process. Here, 0 maydenote split presence information of the 64×16 block (2).

Also, all split information generated during a process of acquiring a64×16 block including only the actual image data from an initial blockor a basic encoding block may be 0 in FIGS. 18A and 18C. However, 0 maydenote different pieces of information.

Also, since both of the cases of FIGS. 18A and 18C may be used to reachthe block including only the actual image data from the initial block orthe basic encoding block, a method using a smaller number of splits maybe implicitly determined. In other words, the asymmetric binarytree-based split method having a smaller number of splits according toFIG. 18C may be determined.

According to an embodiment, the asymmetric binary tree-based split maybe vertically performed when the horizontal length of the regionincluding the actual image data is equal to ¼ or ¾ of the horizontallength of the block before the split on the right boundary of thepicture, the symmetric binary tree-based split may be verticallyperformed when the horizontal length of the region is equal to ½, andthe symmetric or asymmetric binary tree-based split may be verticallyperformed when the horizontal length of the region is equal to ¼, ½, or¾.

More specifically, with respect to the case in which the horizontallength of the region is not equal to ¼, ½, or ¾, a split based on anasymmetric binary tree having a wide right side may be verticallyperformed when the horizontal length of the region including the actualimage data is less than or equal to ¼ of the horizontal length of theblock before the split, and the symmetric binary tree-based split may bevertically performed when the horizontal length of the region is greaterthan ¼ and less than or equal to ½. Also, a split based on an asymmetricbinary tree having a wide left side may be vertically performed when thehorizontal length of the region is greater than ½ and less than or equalto ¾, and the symmetric binary tree-based split may be performed whenthe horizontal length of the region is greater than ¾.

Also, the asymmetric binary tree-based split may be horizontallyperformed when the vertical length of the region including the actualimage data is equal to ¼ or ¾ of the vertical length of the block beforethe split on the lower boundary of the picture, the symmetric binarytree-based split may be horizontally performed when the vertical lengthof the region is equal to ½, and the symmetric or asymmetric binarytree-based split may be horizontally performed when the vertical lengthof the region is not equal to ¼, ½, or ¾.

More specifically, with respect to the case in which the horizontallength of the region is not equal to ¼, ½, or ¾, a split based on anasymmetric binary tree having a wide lower side may be horizontallyperformed when the vertical length of the region including the actualimage data is less than or equal to ¼ of the vertical length of theblock before the split, and the symmetric binary tree-based split may behorizontally performed when the horizontal length of the region isgreater than ¼ and less than or equal to ½. Also, a split based on anasymmetric binary tree having a wide upper side may be horizontallyperformed when the vertical length of the region is greater than ½ andless than or equal to ¾, and the symmetric binary tree-based split maybe performed when the vertical length of the region is greater than ¾.

The above description is associated with some boundaries, and thus asplit on the other boundaries may be as follows.

The asymmetric binary tree-based split may be horizontally or verticallyperformed when at least one of the horizontal and vertical lengths ofthe region including the actual image data is equal to ¼ or ¾ of acorresponding one of the horizontal and vertical lengths of the blockbefore the split on the lower right boundary of the picture, thesymmetric binary tree-based split may be horizontally or verticallyperformed when at least one of the horizontal and vertical lengths ofthe region is equal to ½, and the symmetric or asymmetric binarytree-based split may be horizontally or vertically performed when thehorizontal and vertical lengths of the region are not equal to ¼, ½, or¾. A detailed description of the case in which the horizontal andvertical lengths of the region are not equal to ¼, ½, or ¾ may bederived from the right side or the right boundary and thus will beomitted.

In other words, the symmetric binary tree-based split or the asymmetricbinary tree-based split may be possible on the lower right boundary ofthe picture. Also, on the lower boundary and the right boundary of thepicture, the symmetric binary tree-based split or the asymmetric binarytree-based split may be possible in a split direction parallel to theboundaries and may not be possible in a split direction perpendicular tothe boundaries.

The above-described embodiments of the present invention may assume thatthe block before the split includes both of actual image data and apadded image, and thus cannot be applied to a case in which only theactual image data is included. Also, for convenience of description, theasymmetric binary tree-based split has been described as split into twoparts in the above-described ratio (1:3 or 3:1), but another ratio mayalso be possible depending on encoding settings.

4-3. Adaptive Image Data Processing Method 3 According to a Plurality ofSplit Methods

Here, splits based on multiple trees such as a quad tree, a binary tree,and a ternary tree may be supported for a block, and the adaptive imagedata processing method 3 may be the same as the adaptive image dataprocessing methods 1 and 2 with respect to the other aspects. Also, itis assumed that the exceptional block split settings described in theabove-described adaptive image data processing methods 1 and 2 accordingto the plurality of split methods may be applied to the adaptive imagedata processing method 3.

According to an embodiment, when at least one of the horizontal andvertical lengths of the region including the actual image data is equalto ¼ or ¾ of a corresponding one of the horizontal and vertical lengthsof the block before the split on the lower right boundary of thepicture, the ternary tree-based split may be horizontally or verticallyperformed. Here, when the horizontal and vertical lengths of the regionare not equal to ¼ or ¾, the horizontal and vertical lengths of theregion including the actual image data are compared to the horizontaland vertical lengths of the block before the split. When both of thehorizontal and vertical lengths of the region exceed or do not exceed ½,the quad tree-based split may be performed. When only one of thehorizontal and vertical lengths of the region exceeds ½, the binarytree-based split may be horizontally or vertically performed.

Also, when the horizontal and vertical lengths of the region includingthe actual image data are equal to ¼ or ¾ of the vertical length of theblock before the split on the lower boundary of the picture, the ternarytree-based split may be horizontally performed. When the horizontal andvertical lengths of the region are not equal to ¼ or ¾, the binarytree-based split may be horizontally performed.

Also, when the horizontal and vertical lengths of the region includingthe actual image data are equal to ¼ or ¾ of the horizontal length ofthe block before the split on the right boundary of the picture, theternary tree-based split may be vertically performed. When thehorizontal and vertical lengths of the region are not equal to ¼ or ¾,the binary tree-based split may be vertically performed.

In other words, the quad tree-based split, binary tree-based split, andternary tree-based split on the lower right boundary of the picture maybe possible in a horizontal or vertical direction, and the binarytree-based split and the ternary tree-based split on the lower boundaryand the right boundary of the picture may be possible in a splitdirection parallel to the boundaries. In some cases, however, on thelower boundary and the right boundary of the picture, the quadtree-based split may be possible, but the binary tree-based split andthe ternary tree-based split may be impossible in a split directionperpendicular to the boundaries.

Various embodiments (4-1, 4-2, and 4-3) of the adaptive image dataprocess method according to the plurality of split methods have beendescribed, but may be changed according to block split settings. Thus,the present invention is not limited thereto. In other words, the blockmay have acquirable blocks determined according to the block splitsettings, and thus it is possible to adaptively process block splitinformation on a boundary of a picture.

FIG. 19 is a flowchart showing a method of encoding an image throughadaptive image data processing according to an embodiment of the presentinvention.

Referring to FIG. 19 , an image encoding apparatus for adaptivelyprocessing image data according to an embodiment of the presentinvention may set size information of a picture and size information andblock split information of a basic encoding block (S1910) and expand thesize of the picture such that the size of the picture is an integralmultiple of the size of the basic encoding block (S1920). Also, theimage encoding apparatus may partition the expanded picture into basicencoding blocks (S1930), encode the basic encoding blocks according toblock position information and (re)set block split settings (S1940), andtransmit a bitstream corresponding to the encoding to an image decodingapparatus.

Also, the image decoding apparatus according to an embodiment of thepresent invention may operate corresponding to the above-described imageencoding apparatus. In other words, the image decoding apparatus mayrestore the size information of the picture and the size information andthe block split information of the basic encoding block on the basis ofthe bitstream received from the image encoding apparatus and may expandthe size of the picture so that the size of the picture is an integralmultiple of the size of the basic encoding block. Also, the imagedecoding apparatus may partition the expanded picture into basicencoding blocks and may decode the basic encoding blocks according to ablock scan order, the block position information, and the (re)set blocksplit settings.

Here, the expansion of the picture may be performed in some directions,such as rightward or downward from the picture, according to an integralmultiple of the basic encoding block. The basic encoding block locatedon a boundary of the picture may be adaptively encoded according to thelocation of the block and the block split settings. Here, depending onencoding settings, the block split settings may be used with no changeor be reset with a partial change.

According to an embodiment, split information of a basic encoding blocklocated inside the picture may be explicitly processed, and informationof a basic encoding block located on a boundary of the picture may beexplicitly or implicitly processed on the basis of actual image data.Here, when settings such as the size and form of an acquirable block arechanged and reset according to a split, the settings may be reflectedand then processed.

While the example embodiments of the present invention and theiradvantages have been described in detail, it should be understood thatvarious changes, substitutions and alterations may be made hereinwithout departing from the scope of the invention.

What is claimed is:
 1. An image decoding method, comprising: dividing afirst coding block to determine a second coding block; and decoding thesecond coding block, wherein the first coding block is divided based onone of division types pre-defined in an image decoding apparatus,wherein the division types include at least one of a quad-tree division,a binary-tree division, or a ternary-tree division, wherein thequad-tree division is representative of a division type of dividing onecoding block into four coding blocks by dividing a width and height ofthe one coding block in half, wherein the four coding blocks have thesame size, wherein the binary-tree division is representative of adivision type of dividing one coding block into two coding blocks,wherein the ternary-tree division is representative of a division typeof dividing one coding block into three coding blocks, wherein adivision type for the first coding block is determined to be one of thequad-tree division, the binary-tree division, or the ternary-treedivision when a shape of the first coding block is square, and thedivision type for the first coding block is determined to be thebinary-tree division or the ternary-tree division when the shape of thefirst coding block is non-square, wherein the binary-tree division andthe ternary-tree division are applied only when the quad-tree divisionis no longer performed, wherein the second coding block is determinedbased on predetermined division information, wherein the divisioninformation includes at least one of a first flag indicating whether todivide the first coding block, a second flag indicating a divisiondirection, or a third flag indicating the division type, wherein thesecond flag equal to a first value indicates that the first coding blockis divided in a horizontal direction, and the second flag equal to asecond value indicates that the first coding block is divided in avertical direction, wherein the third flag equal to a first valueindicates that the binary-tree division is performed, and the third flagequal to a second value indicates that the ternary-tree division isperformed, wherein, when the first coding block is located at a boundaryof a current picture, only at least one of the quad-tree division or thebinary-tree division is allowed, wherein the binary-tree divisionincludes a horizontal binary-tree division and a vertical binary-treedivision, and wherein, when the boundary of the current picture is abottom boundary, only the horizontal binary-tree division among thehorizontal binary-tree division and the vertical binary-tree division isallowed for the binary-tree division.
 2. An image encoding method,comprising: dividing a first coding block to determine a second codingblock; and encoding the second coding block, wherein the first codingblock is divided based on one of division types pre-defined in an imageencoding apparatus, wherein the division types include at least one of aquad-tree division, a binary-tree division, or a ternary-tree division,wherein the quad-tree division is representative of a division type ofdividing one coding block into four coding blocks by dividing a widthand height of the one coding block in half, wherein the four codingblocks have the same size, wherein the binary-tree division isrepresentative of a division type of dividing one coding block into twocoding blocks, wherein the ternary-tree division is representative of adivision type of dividing one coding block into three coding blocks,wherein a division type for the first coding block is determined to beone of the quad-tree division, the binary-tree division, or theternary-tree division when a shape of the first coding block is square,and the division type for the first coding block is determined to be thebinary-tree division or the ternary-tree division when the shape of thefirst coding block is non-square, wherein the binary-tree division andthe ternary-tree division are applied only when the quad-tree divisionis no longer performed, wherein the second coding block is determinedbased on predetermined division information, wherein the divisioninformation includes at least one of a first flag indicating whether todivide the first coding block, a second flag indicating a divisiondirection, or a third flag indicating the division type, wherein thesecond flag equal to a first value indicates that the first coding blockis divided in a horizontal direction, and the second flag equal to asecond value indicates that the first coding block is divided in avertical direction, wherein the third flag equal to a first valueindicates that the binary-tree division is performed, and the third flagequal to a second value indicates that the ternary-tree division isperformed, wherein, when the first coding block is located at a boundaryof a current picture, only at least one of the quad-tree division or thebinary-tree division is allowed, wherein the binary-tree divisionincludes a horizontal binary-tree division and a vertical binary-treedivision, and wherein, when the boundary of the current picture is abottom boundary, only the horizontal binary-tree division among thehorizontal binary-tree division and the vertical binary-tree division isallowed for the binary-tree division.
 3. A non-transitory computerreadable medium storing a bitstream generated by an encoding method, themethod comprising: dividing a first coding block to determine a secondcoding block; and encoding the second coding block, wherein the firstcoding block is divided based on one of division types, wherein thedivision types include at least one of a quad-tree division, abinary-tree division, or a ternary-tree division, wherein the quad-treedivision is representative of a division type of dividing one codingblock into four coding blocks by dividing a width and height of the onecoding block in half, wherein the four coding blocks have the same size,wherein the binary-tree division is representative of a division type ofdividing one coding block into two coding blocks, wherein theternary-tree division is representative of a division type of dividingone coding block into three coding blocks, wherein a division type forthe first coding block is determined to be one of the quad-treedivision, the binary-tree division, or the ternary-tree division when ashape of the first coding block is square, and the division type for thefirst coding block is determined to be the binary-tree division or theternary-tree division when the shape of the first coding block isnon-square, wherein the binary-tree division and the ternary-treedivision are applied only when the quad-tree division is no longerperformed, wherein the second coding block is determined based onpredetermined division information, wherein the division informationincludes at least one of a first flag indicating whether to divide thefirst coding block, a second flag indicating a division direction, or athird flag indicating the division type, wherein the second flag equalto a first value indicates that the first coding block is divided in ahorizontal direction, and the second flag equal to a second valueindicates that the first coding block is divided in a verticaldirection, wherein the third flag equal to a first value indicates thatthe binary-tree division is performed, and the third flag equal to asecond value indicates that the ternary-tree division is performed,wherein, when the first coding block is located at a boundary of acurrent picture, only at least one of the quad-tree division or thebinary-tree division is allowed, wherein the binary-tree divisionincludes a horizontal binary-tree division and a vertical binary-treedivision, and wherein, when the boundary of the current picture is abottom boundary, only the horizontal binary-tree division among thehorizontal binary-tree division and the vertical binary-tree division isallowed for the binary-tree division.